"Sunday lunch with family" and "Sunday lunch with the family" are both correct, and have slightly different meanings. It's not a grammar question, it actually illustrates the meaning of the English word "the." If my friend Dave says "I am having Sunday lunch with family" he is not being definite. He is not telling me, nor can I guess, exactly whom he means. It might be his wife and kids, it might be his Aunt Vera and Uncle WIll, it might be his cousins in Springfield... I don't know. "Family" here is being used as an uncountable "mass noun." What KIND of people is he lunching with? Not with strangers, not with friends, but with family. If next week he says "I'm having Sunday lunch with family again," it might be a completely different set of people. If Dave says "I'm having Sunday lunch with THE family" he is being definite. I can probably guess who is going to be sitting at the table. In U.S culture he probably means "my 'nuclear' family." If Dave is forty, he probably means his wife and kids. If Dave is twenty, it could mean his parents, brothers, and sister. Either way, he means "THE family," a SPECIFIC family, a DEFINITE family. If next week he says "I'm having Sunday lunch with the family again," it is the SAME family, and the same people as before will be sitting around the table.

Timing should be supposed to be placed on the last position in an English sentence I remember. New year days to stay with family in China every year.