Yes, it does happen. I don't think it's because you're more aware of your mistakes. I think it's because:
1) we expect more. We think that all that time should produce immediate, noticeable results. Our expectations that we will see instant change and that everything follows a clear cause-and-effect pattern disappoint us. Of course an intense weekend of study isn't going to result in miraculous improvement overnight. Improvement is cumulative and you notice the "shocking" changes weeks after you began. It's like that friend who got healthy. They still looked flabby despite working out regularly and not eating junk three weeks in, but months later, they look like a different person. They were changing all along, but too slowly for it to be noticeable. It's similar with a new language.
2) being in "monitor mode" where you're constantly thinking about the rules and possibly over-applying new knowledge makes you second-guess yourself. Instead of just letting the words tumble out naturally (and often correctly!), you're stuck. Hypervigilance does not work for me when it comes to speaking another language. At most, it allows me the feeling of being virtuous (what a good girl I am, looking out for errors and avoiding saying anything imperfect) but doesn't really help fluency.