I think it depends on what you want to achieve by learning the language :)

If you plan on visiting the country, then everyday expressions, and phrases for getting around and such would be helpful.
Should you be interested in studying something specific (for example, Japanese music), then you'd need to work on reading/listening so that you can gain more knowledge in the area.
If you're studying just for fun, then maybe easy conversation starters (simple questions, basic sentences) might be best for your next step - in which case you would have to start with grammar.