Yes.
If there is a comma (nonrestrictive clause), you HAVE TO use which.
If there isn't a comma (restrictive clause), "which" and "that" are both OK, but "that" is better.
1. This is the office building, which has 2 lunch rooms.
2. This is the office building that has 2 lunch rooms.
1= There is only one office building. The part after "which" is giving extra information that doesn't change the primary meaning of the sentence.
"This is the school, this is the library, and this is the office building, which has 2 lunch rooms."
2=There are several office buildings, and you are identifying one of them. The part after "that" is necessary for the sentence.
Please go to the office building that has 2 lunch rooms. Don't go to the office building that has 3 lunch rooms.