I think this question doesn't have an obvious answer because each learning steps have their own challenges. For me I think that speaking is the most challenging part but it will be different for all people. Personalities do affect the learning process. I'm an introvert and I'm not a talkative person. So, the speaking part is quite difficult for me.