Observe the following examples:
Affirmative Question: Do you like spaghetti?
Affirmative Answer: Yes, I do. (I like spaghetti.)
Negative Answer: No, I don't. (I don't like spaghetti.)
Negative Question: Don't you like spaghetti?
Affirmative Answer: Yes, I do. (I like spaghetti.)
Negative Answer: No, I don't. (I don't like spaghetti.)
When answering a question answerable by yes/no, the answer does not totally rely on the question. If your answer is negative you simple say No + another negative remark (No, I don't), and if affirmative you say Yes + positive remark (Yes, I do.)
This may be confusing for non-native English speakers because in other languages, the response relies on the question... like in Korean, if you ask them a negative question, they will say Yes first then say a negative remark to mean they agree to the negative question (that their answer is also negative.
Just remember in English, we say Yes + positive remark (for positive answers) and No + negative remark (for positive answers).
I hope this helps.