You should've told us the method you've been using to increase your vocabulary; otherwise we cannot provide suitable options as we are unaware of your methods.
In my experience, one of the best ways to improve your lexical repertoire, and, by extension, to increase your overall fluency is to read as much as you can about a myriad of topics, as well as to talk to yourself and write summaries or essays about said topics.
We have two databases in our brains, one for passive vocabulary and another one for active vocabulary. When you read you are memorizing new words only if you're paying enough attention, but those words may only be stored in your short-term memory, the only way to store them in your long-term memory and never forget them is by actively using the words you're trying to learn. Hence my suggestion to talk to yourself a lot and write about what you read. You need to force yourself to use the language, otherwise you will constantly forget everything you learn. Of course, there is also the possibility that you have memory issues in general, if that happens to be the case, a doctor would be of better help.