Native speakers may not be good judges of how difficult their own language is. Monolingual native English speakers (who mostly don’t interact with multilingual people) tend to think English is difficult to learn. Most learners of English don’t agree.

As someone who only just started learning Chinese, who is still at the pre-beginner level, I think that there are aspects that are more difficult (distinguishing tones in listening and especially speaking, using pinyin as a pronunciation guide), medium difficult (learning about radicals and identifying different characters), and not difficult once you get used to it (grammar and sentence structure).

Understanding each character represents a syllable (including tone) was what allowed me to get the basic hang of things.

I've been learning Mandarin as a native English speaker for the past year, I can confirm it is a bit difficult at the beginning (at least for me it was). The most difficult parts in my experience were learning Chinese characters, (there is no alphabet), and learning the tones and when the tones change. It's been a lot of fun though and it definitely has been useful for me but that's partially because I live in the Vancouver area. If you have any specific questions feel free to ask me!