Definitely "I forgot to tell you". And the title of your question should read, "Which is correct", instead of "What is correct".

I would say "I forgot to tell you". That is because you are with the person at that moment and are telling them ... so the situation of forgetting is finished. You could use "I have forgotten" with other people. For example, "I have forgotten to tell my wife I will be late"

For 'I have forgotten to tell you', usually we say 'I've forgotten to tell you', which sounds more natural. The second sentence is correct to so concluding, both sentences are correct and used often in conversations Hope this helps!