It depends when and where you begin. For babies, all languages are easy. :-) For adults and teenagers from an English (or sometimes Spanish) language background, the US Defense Language Institute divides languages into four categories (https://www.ausa.org/articles/dlis-language-guidelines). The hardest category includes Arabic, Korean, Japanese, Mandarin (Chinese), and Pashto. I've no doubt that Cantonese, Hokkien, and all other Chinese languages would also belong in that group. I understand that Burmese is also very difficult (it has tones like Mandarin, politeness changes like Korean, and its own alphabet). But obviously, this all depends where you begin. When I was studying Mandarin, I had a classmate who was fluent in spoken Cantonese (and could read some Chinese characters). She learned much, much more quickly than everybody else. Every time the teacher asked "do you understand?", my Cantonese-speaking neighbour would say "tingdong!" ("yes!"), and the teacher would start the next point. ;_; ;_; ;_; And there are also some languages where there are very few textbooks and other learning materials. A friend of mine is learning the Ashe language from Nigeria for his research work. There's no Ashe-English dictionary. In fact, there are no Ashe dictionaries at all! So that is a very difficult language for everybody, unless you are very lucky and already speak a nearby (Plateau) language.

Chinese is one of the most difficult languages. I study Chinese, but it's very difficult... extremely difficult. Anyway it concerns only writing aspect. Grammar is easy enough

As a beginner in Mandarin myself, I think it's the toughest language I've ever stumbled upon. I mean, I need to memorize a great deal of characters including its special tonal and context variations which requires my undivided attention. Hundreds or even thousands characters, seriously?

It depends on your native language. It is simply harder to acquire some languages depending on your current skills. Generally, I imagine tribal languages are difficult because they have no commonalities with other languages. I thought Mandarin was hard until I tried Mohawk!

It depends. It's subjective think. In my opinion the most difficult are languages of northern Europe. I mean German, Dutch, Flemmish, and all scandinavian languages.