For me it would have to be the pronunciation and different phonetic aspects of the language, its seemingly arbitrary way on how you pronounce certain words one way and some others in a totally different manner even though they share the same vowels and consonants.
Other than that I think English is one of the easiest languages to learn.