There is only ONE obstacle to learning a language: laziness.

It's very difficult when they are talking something very native, I mean, you need to know some background to understand them. For example, "She is like *** (a character in Simpsons)". It's hard to understand what kind of person is she like because we may not familiar with their culture.

I am an English speaker but I will say that the only obstacle I have really noticed is when Americans and other native english speakers speak we tend to "change the meanings of things, for example: The phrase "are you nuts" may make no sense to a forigner but to Americans it is basically asking if a person is crazy. I wouldn't think the American language is hard but practice makes perfect. I am learning Japanese at the moment and that is a challenge. Just keep to it and work hard.

Can you speak Mandarin Chinese?