For some people, it can be difficult because their brain isn't "wired" to learn languages. Just like some people can learn to play an instrument, even without formal training or the ability to read musical notation, and others struggle to learn. We all have different aptitudes.
That being said, a reason for learning can be a big force toward learning. In many parts of the world, the "smart" people see they need to learn another language to be economically successful. So they put in the effort to do so.
In the US, we have the "curse" that English has turned into the language of international business, so many Americans don't feel they "have to" learn another language. Plus, our schools do a very bad job of teaching languages.
So, desire, aptitude, effort all play a big role.