Actually, 'I'm starting (or beginning) to forget him.' would be correct if you are trying to indicate that you don't think about someone as much as you did in the past (perhaps after a break up). Once you have stopped thinking about someone or something you would say 'I've forgotten about him/her/it.' If you are talking about someone who actually is losing memory of a person because that person died when the speaker was a child, you could say: My memories of him are starting to fade. I have few memories of him. I have forgotten a lot about him. I don't remember a lot about him.

It's informal English, so it is not grammatically correct. You can also say, "I'm slowly getting over him."