I took TOEFL years ago with no preparation and I did worst at writing. I thought it was my speling, as you can't use spell check, but later I heard that they expected the thing to be written in a certain way, following certain structure, so maybe it was that, or who knows.

In my opinion, the most difficult part in TOEFL exam is speaking section because it's hard to manage the time to speak while you are recording your voice. Sometimes you can't answer all questions or forget some points to talk about. This is what I faced in the test.

Hi, for me, the speaking part is the hardest. You have to talk to machines..... That is not natural. I like IELTS better regarding speaking section.

In my opinion, writing section is the hardest one. 

Thanks so much for your responses! Speaking and Writing are the most difficult parts of the test for lots of students, for exactly the reasons you mention. I'll write directly about the specifics of your responses.