Actually, neither of them are correct :)
The word you are looking for here is "closed" not "shut". The meanings are very similar, but we use closed and not shut for these sentences.
I think the correct sentence is 1, but both of the sentences are correct. It just depends on the situation:
1. I wouldn't have gone to the museum if I had known it was closed
This means that you arrived at the museum, and once you got there, you saw that it was closed.
2. I wouldn't go to the museum if I knew it was closed.
This is more of a general statement. It also sounds like you might be defending yourself against an accusation. Most likely this is not the sentence you need.