Use the proposition "with," and use the possessive pronoun "my" before "family:" "I like to spend time with my family." "I want to spend more time with my family." As a matter of fact, when someone important leaves a job, and nobody wants to tell the press the real reason, the standard thing they say to the news media is "I left so that I could spend more time with my family." Here are some examples from recent news stories. They show that that "spend time with my family" is a common expression. “It was a very hard decision,” [FDA commissioner] Gottlieb said in an interview. “This is the best job I will ever have. I’m leaving because I need to spend time with my family." Pakistan cricket player Shoaib Malib said "I announce with mixed feelings that I will be not be part of T10League to spend time with my family." "[Talk show host] Julie Chen made it official Tuesday. She's leaving 'The Talk' because 'I need to spend time with my family.'"

It sounds more natural to stay: "be with my family" "spend some time with my family"