In both cases you have to use “months” The meaning changes though: “few” specifically means very little, often less than what you expect. “A few” simply means “not many” I started speaking Italian after only few months of study. After a few months my writing began to improve as well.

After a few months

using "few" without "/a/ few" is mostly used to indicate an emphasis on how little it is. Most of the time you should probably be using "a few" unless you're specifically trying to emphasize the negative nature of some amount. As a measurement of time "a few months/ a few hours/ a few weeks/ days/years/minutes etc" you should pretty much always use "a few" because you're actually trying to talk about the measurement of things. It ends up sounding kind of weird even as an emphasis of a short time UNLESS you use some adverb with it "I started speaking English after VERY few weeks/months of study" that sounds more normal and natural. This is a difficult question in general though and I'm sure plenty of people would have other things to say about it.