A fifth wheel is someone who doesn’t belong in a group, regardless of the size of a group. A third wheel is an unwelcome third person, usually a single tagging along with a couple.

The car normally has 4 wheels, that's why "the 5th wheel" is a common expression. "The third wheel" is a very rare expression, if at all.

A "third wheel" is someone accompanying a couple. The couple may want the 3rd person along, but the 3rd person may feel uncomfortable and out of place with a couple and so considers himself/herself as a "3rd wheel". This is a negative description. Someone outside of this group of three may also describe this 3rd person as a 3rd wheel in a mocking or deriding manner because THEY feel this person is out of place. What I mean is, the couple often don't consider the person with them as a 3rd wheel (because it has a somewhat negative meaning). Usually it is the 3rd person themself feeling out of place OR someone totally different being a little nasty to that 3rd person. Fifth wheel has little meaning where I am. You would say "spare wheel" instead of that all the time - and this being the one extra wheel you have in addition to the 4 required for the car.