I prefer #1, but #2 is just as good if you omit some words.
#1 has two interpretations:
a) We went there (Rome, for example), but our purpose for going there was not to visit the Sistine Chapel.
b) We didn't visit the Sistine Chapel.
#2 is the same as:
We didn't visit the Sistine Chapel.
We didn't go to the Sistine Chapel.
We didn't go visit Sistine Chapel.