The second option, "I shouldn't have said that", is the correct one. Using 'have' makes it past tense.

The correct answer would be "I shouldn't have said that" However, I could see you saying "I shouldn't say that" if you say it IMMEDIATELY after the thing that you regret saying. Example - "You're not the best at what you do. ... Sorry, I shouldn't say that."