I agree with Mathias and Abhi: follow the grammar of the question if in doubt. "Yes, you have to (take it)." That said, "yes, you do" is still a common and perfectly acceptable response.

yes you have to

I was said to use the same kind of thing that is used in the question. Here the question has "have to". So, in the answer also "have to". Refer to Matias' answer.

Both are fine.