The natural answer is 'It's a tree'. It would NOT be normal to use 'This' in the answer.
A native speaker would not normally say 'This is a tree' as an answer to 'What's this?'. Here's why:
You use the demonstratives 'this' and 'that' to specify a particular thing. As the question has already specified the object, it would be wrong to specify it again in the answer. Instead, we use the pronoun 'It'.
If the tree is closer to the person who's asking the question than to the person who's answering, you could say 'That's a tree'.
If you said 'This is a tree', it would actually refer to another object. For example, say your friend is standing near a green leafy object and you're standing next to another green leafy object. Your friend points to the one nearer to him and says "What's this?", you could say "Well, that's a tree (referring to the object he asked about) but this is a bush" (referring to the object near you). In other words, 'this' refers to a DIFFERENT object.
I hope that makes sense.