In American English "family" is a fairly loose term, typically meaning your immediate relatives or possibly even people you feel a particularly close connection to and live together or have lived together with in the same household: father, mother, wife, husband, brother, sister, children. "Relatives" are typically people you are connected to by blood or marriage, but don't have the same close relationship with.