It depends on your goals. If you want to be able to converse with people, I would ignore grammar for now, memorize some phrases, and then actually speak with some Icelanders. Grammar is so much easier to learn once you've had some context. But if your goal is to be able to read Icelandic text, grammar is probably more important.
In either case, I'd recommend Beginner's Icelandic by Hilmarsdóttir and Kozlowski. It's a good text to start off with. Once you've finished that, I'd follow up with Colloquial Icelandic, which is a bit intense grammar-wise.