Yes.
"Withdraw" simply means "to take out" or "remove." You withdraw money from a bank.
With regard to a public library, we can say "I withdrew a book from the library," "I borrowed a book from the library," "I took out a book from the library," or simply "I got a book from the library." They all mean the same thing.
The word "withdraw" doesn't really say whether you are going to return it or not. In U.S. English, a library is never a bookstore that sells books. It usually means a place that lends them. Very often, it means "public library." (The United States is fortunate to have many public libraries, in part because of the philanthropy of steel millionaire Andrew Carnegie who made it a personal mission of his). (It can sometimes mean "a room in a house where books are shelved.") The word "library" tells us that they are borrowing the book.