This question is very debatable.
I think it would always depend on who you ask.
But generally speaking, and of course basing on the language's syntax and composition...I would say CHINESE! With the English language, you just have to learn the alphabet and their sounds, put them together and voila!- you have a word..then get on with the correct pronunciation, meanings, and grammar. With Chinese, you have to get accustomed to the tones, pinyin, and memorize the CHARACTERS (correct pronunciation, the number of strokes, the radicals, etc..)!