I have a London accent so I don't have this problem!  However, here are some ideas:

a) pay attention to the context - which one makes more sense?

b) what is the feeling? positive or negative?

c) try to identify the /t/ sound at the end - sometimes it is like the airflow has suddenly stopped in the throat

Don't worry if someone speaks fast and you can't hear exactly if it's supposed to be <em>can </em>or <em>can't. </em>It's normal to ask people to repeat and emphasize on which word they're saying.

I agree with Stephanie. If you're familiar with IPA, I usually hear <em>can </em>being spoken with the ə sound (like the 'a' in 'about') in American English. <em>Can't </em>sounds more like the word <em>can </em>(the tin container, not the verb) or <em>tan</em>.

for can: a little bit longer, a little smoother, versus can't, can't: a little shorter, a little bit more abrupt

http://rachelsenglish.com/pronounce-can-vs-cant/ ;