As nouns, there is little difference. "Candy" is more common in the U.S. but both are used. Note that they are handled differently: Would you like some candy? Would you like a piece of candy? Would you like a sweet? Would you like some sweets? I don't want candy. (or "any candy") I don't want sweets. (or "a sweet")

American English-candy British English-sweet

As Glenn says, 'candy' is a noun and 'sweet' is an adjective. However, in British English, 'sweet' can also be a noun. It is what some people call 'dessert' so, for example, apple pie and cream can be a sweet. We also call things like Haribo 'sweets'.