Unfortunately, there is no one simple answer to this question. There are some languages that have a number of characteristics that make them relatively difficult to learn. However, it depends much more on what languages you already know, and what language you grew up speaking. Certainly, having no prior experience of learning languages will make the journey a little more difficult for you. Similarly, learning a language in your home country without the assistance of a native speaker presents many difficulties as compared to immersion learning, of course.
Your native language
The language you were surrounded with when you were growing up (or languages, for those lucky multilingual few) is the most influential factor for how you learn others. Languages that share some of the qualities and characteristics of your native tongue will be easier to learn, while languages that have very little in common with it will be much harder. Most languages fall somewhere in the middle.
Learning a language that is closely related to your native language, or another one that you already speak, is much easier than learning a completely alien one. Related languages share many characteristics and this tends to make them easier to learn being that there are less new concepts to deal with.
One of the characteristics that is often shared between languages is grammar. In Swedish, word order and verb conjugation is mercifully similar to English, which makes learning it much easier than say German, which has a notoriously more complex word order and verb conjugation. Although both languages are related to English, German has retained its more complex grammar, whereas English and Swedish have largely dropped it.
The “romance languages” (French, Spanish, Italian, Portuguese and a number of others) are famous for sharing many characteristics. It is not surprising since they all evolved from Latin. It is very common for someone who learns one of these languages to go on and learn one or two others. They are so similar at times that it seems that you can learn the others with substantially less effort.
Cognates and borrowed vocabulary
This is one of the characteristics that makes the “romance languages” so similar, being that they all get the vast majority of their vocabulary from Latin. Similarly, English has also borrowed much of its vocabulary directly from Latin and what it didn't get there, it just borrowed from French. In fact, there is an enormous amount of French vocabulary in English. This is one of the reasons why English speakers consider Spanish, French and Italian to be easier to learn than other languages.
Obviously, languages sound different. Although all humans use basically the same sounds, there always seems to be some sounds in other languages that we just don't have in our native one. Some are strange or difficult to articulate. Some can be quite subtle. A Spanish o is not exactly the same as an English o, for example. Moreover, there are some vowel sounds in French that just don't exist in English. On the other hand, while a French r is very different from English, a Chinese r is actually very similar.
It can take some time to get comfortable with these new sounds, though I think that faking it is acceptable until you can get a better handle on them. Many people don't put enough effort into this aspect of learning and this makes some languages seem harder to learn than they need to be.
Some languages actually have aspects of the culture built right into them. In English, for example, we can address a stranger in much the same way as we would our friends. However, in many European languages, you actually need to use a different form of “you” and a different verb conjugation depending on the person you are talking to, even though you are saying the same thing.
In Japan, these levels of respect are taken to the extreme. You will need to use different words to say the same thing depending on whether you're talking to your sister, a stranger, your boss or the President. This kind of subtlety can be very tricky and certainly adds to the difficulty of the language.
To summarize, there are many factors that make one language harder or easier to learn. First of all, your native language will influence this. However, there are also some other concrete factors, such as your intelligence and natural talent for languages, although many of these can be learned or improved with training. Furthermore, individual limitations in these areas can be offset by learning how to learn languages. Master the unique skill set that language learning requires and anyone can improve their ability to learn languages.
So, is Spanish an easy language to learn? Well, if your native language is English then, yes, it is relatively easy as compared to some other languages: you use the same writing system, and the pronunciation, grammar and vocabulary are all relatively similar. If your native language is Italian, it's even easier. However, if your native language is Chinese, then it's probably not easy at all. There are very few recognizable traits that you will find in Spanish.
Some languages have additional levels of complexity over others and this certainly makes some inherently harder or easier than others. However, your native language will always influence and “colour” your learning process. Even so, you needn't be limited by it. Learn and absorb like you did when you were a child and “easy” or “hard” will become irrelevant.
And one last thought before I finish: let’s make this the year that you commit to learning a foreign language! Together we can do it!!