Neither is correct! Correct is "What does it mean?".

The grammar here is incorrect. "What does this mean?" "What does that mean?" Use this if you're showing the person something and asking what it means (Or if it's very close to you in relation to the other person). Use that when you're asking about something that has already been put introduced or is relatively away from you) Examples: Student A is reading a text book and comes across a term he doesn't understand. Student A slides their textbook to Student B, pointing at the term and says "What does this mean?" Teacher: Usually, a system will not be in an eigenstate of whatever observable we are interested in. However, if one measures the observable, the wavefunction will instantaneously be an eigenstate of that observable. This process is known as wavefunction collapse. Student: What does that mean?