if you say" I wasn't listen to him " its not correct. I has to be "I wasn't listening to him" if you want to use listen you can say "i did not listen to him" one last thing to note is that there are some verbs that require one form or the other, and that will overrule any other consideration "I enjoy listening to music" [correct] "I enjoy to listen to music" [wrong] and "I want to listen to music" [correct] "I want listening to music" [wrong]