Both of the answers could be correct. If I am talking about about and care about whether they had seen the movie at some time in the past, you have to say they “had not seen” Example. In 1975 my aunt and uncle took me to see “Butch Cassidy and the Sundance Kid”. Before going, they told me that they hadn’t seen it yet. (It was their first time. Now I know that they have seen the movie, because they saw it with me!) But I could be talking about the present. I care whether they have seen it now. In that case you could say either depending on the timing. Example. Our daughters replied to my text asking if they had seen the new Mulan. Text 1: Have you seen the new Mulan? Text 2: No. We’d like to watch it with you. Or No, we haven’t seen it yet. We’d like to watch it with you. I could say They said they haven’t seen it yet and want to watch it with us. (If the text is in the recent past, two minutes ago for example, I can confidently make a statement about the present.) If the text exchange took place two days ago, I might not be as certain about the current situation and might say ‘They hadn’t seen it and wanted to watch it with us’.

Both are correct. The two options represent two different tenses...

Whoops! Sorry about that. “Said” is past tense so you WOULD use had not to agree with the tense of said.

Hello! It is they have not seen the movie yet!