I frankly wouldn't draw any serious distinctions between them in this sentence. Mind that there are other differences in usage in other sentences, I suppose, but vocabulary building is the hardest part of language learning and to worry about tiny differences like the one you mentioned in this sentence is unnecessary. Go onto the next word or phrase and forget this one. It's fine either way. No native speaker would misunderstand you, whichever one you used, or notice a problem with it.

In this context, the correct answer would be, "I don't know how she failed the exam." The speaker doesn't understand how it is possible since she is good at math. It's possible to say, "I don't know why she failed the exam," but it's a little ambiguous because it could also mean that she deliberately failed the exam for some unknown reason. It's not as common as 'how' in this context. A better use of 'I don't know why' would be like: "I don't know why she doesn't like me" or "I don't know why he killed himself" or "I don't know why this music makes me want to dance."

I did not know, what I know is that she did it. She is always good at math!