Generally, you say... Answer me, please if you are asking someone and only you both are in the picture. On the other hand, I can think of an example of Answer to me Suppose you found one of your employees culprit. The scene includes the employee, boss, a sweeper and you. If you are asking something and if he's answering a sweeper, you can draw his attention by telling him, Answer to me (not him). However, you put an accent on the pronoun 'me' there. Answer to [someone] also works when the context is 'reporting' to someone. It simply means, that someone is the boss as DavidRicherby gives his input. John answers to Jane says that John reports to Jane as she's the boss. Also, you can say "my answer to him was no" but then, it's an all different situation. https://ell.stackexchange.com/questions/40538/word-usage-answer-me-or-answer-to-me

answer the question, and answer the teacher/ answer me! (provide/give an answer to the question) :)