I could give you 101 answers to that question, I think :)
Of course any student's existing language skills are an important factor. For speakers of European languages, Arabic is particularly unfamiliar, just like Chinese or Korean. The Arabic grammar is extensive, complex and at times completely illogical, at least to me.
This article provides a interesting insight in some of the common frustrations among students of Arabic: http://www.slate.com/articles/arts/culturebox/2005/06/im_trying_to_learn_arabic.html
Personally, I don't think the alphabet is the problem. It's always tricky to learn a new script, but I've never felt that the Arabic alphabet is harder than the Latin or Cyrillic or Greek ones. I've never even started to learn Chinese writing, but judging from what I've heard about it, it would be far harder to learn.