- The person you are referring to is me.
- The person who you are referring to is me.
Both of those sentences are correct in the sense that people actually talk that way.
The most correct structure would be "The person to whom you are referring is me." No one in the United States would ever say it like that.
"Are you referring to me?" is how I would ask the question. "Am I the person you are referring to?" is another possibility.