A "celebrity" means anyone who is "celebrated," although that meaning of "celebrated" is old-fashioned and not used much. "Celebrated" literally means "honored," but it usually means "celebrated far and wide."
That is to say, "celebrated" is a synonym for "famous."
There's no precise distinction, but "celebrity" suggests a less elevated status; temporarily famous, famous in magazines rather than history books, famous in pop culture.
Christina Kirchner is "famous," Miley Cyrus is merely "a celebrity."
If someone is on a magazine cover, but we don't know his or her name, then they are not "a celebrity."