Yes, it's probably just a lack of practice. Sounds you are not used to recognise. (I understand because we I speak English I fail to hear the letter H because we don't pronounce it in French)
For me, "les gens" is miles apart from "les ans". (The first one sounds "lè jen" and the second "lè zen".) G is pronounced with top teeth almost touching bottom teeth in mock growling look. Z is a tongue only noise. Train your ears, then your tongue ! Good luck.