I agree with Katerina about the diglossia of Arabic.
Choosing between a British accent and an American accent doesn't compare to a learner of Arabic having to learn a spoken dialect in addition to Modern Standard. Arab nationals studying English literature complain about having to read Shakespeare. Welcome to our world. Except learners of English aren't expected to know Shakespearean English, whereas students of Arabic are expected to know MSA.
Lizzy's point about lack of diacritics in texts and Saqib's comment about the huge vocabulary are also valid points about the difficulty in learning Arabic.
I would also add that a lack of resources in learning Arabic dialects adds to the difficulty, combined with a reluctance on the part of native speakers to speak Arabic with foreigners if they know English.