I would answer 'these' only if there were more than one set. But in the question there is no indication that there are any other keys. There is no need to say 'these' because there are no other options, you are not differentiating between different sets of keys. You are just saying that they - the keys that you are talking about - belong to a particular person.

The only reason I could think of that being marked incorrectly is: "These" implies that the objects you are talking about are very close to you in proximity, or you have possession of them. "They" imply that the objects are further away from you and/or you do not have possession. Since somebody else used "these", it implies that they have possession over the keys. So since you do not have possession, you would respond with "they". In typical American English, I would probably respond with "these" or "those".

Presumably someone has pointed to a single set of keys and it's understood that you are talking about the same ones in both sentences. So once you have identified which they are in the first sentence, there is no need to do so again, and so "they" is usually more appropriate. But there nothing grammatically incorrect about saying "these", and indeed it would be called for together with pointing at them if it still wasn't entirely clear which keys you were talking about after the initial question.

The person who asks the question uses the word 'these', meaning that the keys are closer to him/her than they are to the answerer. The person asking the question is probably holding the keys. This is why it is wrong for the answerer to repeat 'These'. He/she could either say 'Those are mine', indicating the relative position, or 'They are mine' which is neutral and gives no indication of position. I hope that makes sense.