I have had the same problem with Spanish, I have been studying Spanish, off and on, for 15 years, and I can understand a lot that I read, but when I listen to someone speak, I understand very little. Sometimes I have felt like giving up. Recently I have concentrated on a lot of reading and listening, and it seems to help build my vocabulary. I know I need more practice talking to real people, but I'm so bad at it, I don't want to bore the Spanish speakers. I probably make their brain hurt! But maybe if you're bad in English and I'm bad in Spanish we could both drive each other crazy and maybe help each other :)