Age. A dog is a puppy until it is about 6 months to a year old. My 3 year old dog still acts like a puppy, but we refer to her as a dog.

Technically, age is the difference. But in practice people very often refer to dogs of any age (as an affectionate affectation) as "pup" or "puppy". This is done in a similar spirit to the way parents often refer to their children (even grown-up ones) as "my baby".