In many cases, these two words do sound virtually the same, but here is something that might reassure you. I'm a native British English speaker with, I think, a good 'ear' for language, and a number of American friends and colleagues. But if I were to hear an American say the two phrases 'I can' and 'I can't' in isolation, I might struggle to hear the difference.
However, in the real world, words never are spoken in isolation. There is usually another verb following 'can' or 'can't, and the sentence stress is the biggest clue as to whether the statement is positive or negative. If you hear someone sany 'I can't come', the stress will be on the 'can't'. The negative form will be longer and more emphasised. But if the person says 'I can come', the stress will be on 'come'. This means that the word 'can' is shortened and weakened, the vowel becomes a schwa, and the phrase sounds more like " I c'n come'. This difference is crystal clear to any native speaker, American or not.
And even if there isn't a word following 'can' or 'can't, there is always context. For example, if someone says 'Can you come to the party?', and a speaker of American English replies with something that sounds like " I can' ", you will always pick up other clues as to whether this is a negative or positive response. The word may have a slight falling intonation to show regret, or an upbeat feel to show enthusiasm, and their facial expression and body language will probably tell you what they mean. They will probably add 'I'm busy tonight' or 'I'm looking forward to it', which will also confirm whether the response is positive or negative.
In other words, don't just concentrate on the single word - listen to everything around it.