Dan makes a good point. While we can always use “which” as a relative pronoun in reference to a thing (use “who” for people, “where” for places, “when” for times), it is preferable (at least in American English) to use “that” if the clause is defining / restrictive (i.e. the point of the whole sentence). If the word “that” refers to the object of the relative clause, then it may be omitted. If it is the subject, it may not be omitted. (A general rule in English is that the subject is always stated explicitly, except for “you” in the imperative.) Note that “that” cannot be used for a non-defining / non-restrictive clause (i.e. just giving extra information), and that such clauses are always set off by pauses in speech (and commas in writing).

Yeah, both of them are correct. You can also drop "which" & "that".

I'm going to answer from my own thoughts - I'd be interested to see if someone has a more expert opinion. I'd say use THAT in case like this, i.e. where what THAT is referring to is the whole point of the sentence. I'd say use WHICH when it's more of a subordinate clause or side note. Example: The book, which I borrowed from the library, taught me more about woodworking that any I had ever read.

You could just also skip the dilemma and say: "This isn't the book I borrowed from the library." In my opinion, most English speakers just use "that" in any situation and no one cares unless they are like a college professor if the word should be "who" or "which" or "that". That is just my opinion though.