Although Person A's sentence is in past tense, Person B's sentence is not. To understand this we need to think about what "That" (in Person B's sentence) means. You probably know that "that" is a pronoun, and of course a pronoun can replace a noun or noun phrase. So when Person B uses "That" they might mean something like "Watching TV " or "The fact that you had the opportunity to watch TV", and therefore "is" would be the correct choice