If I knew someone's name in the past, but now do not know it (because I have a bad memory!):
Oh dear, I don't remember his name. (Or, can't remember)
Oh dear, I forget his name.
Oh dear, I have forgotten his name.
I think the Americans amongst us would add "Oh dear, I forgot his name." to that list. Please ask an American. However, I do not use the past simple in this case (assuming I am talking to you NOW about someone who I can see now).
At a party I say, "Who is that man? I know his face, but I don't remember his name."
Or, "Who is that man? I know his face but I forget his name."
Or, "Who is that man? I know his face but I've forgotten his name."
If I was talking about a party YESTERDAY, I'd say, "It was so embarrassing! I completely forgot his name. I went to say 'hello, Mr So-and-So and all I could say was, 'Hello Mr... Mr... Mr....."
Or I might use the past perfect on this occasion. "I had forgotten his name!"
So check out the use of the present, present perfect, past simple and past perfect. That is actually a huge area of grammar! Take your time!
Don't forget that in the USA they often use the past simple where the English would use the present perfect. It does not affect understanding. It's the same grammar, but different regions have different preferences and styles. All are valid.