To understand the difference between prepositions it is often helpful to go back to their physical meanings. "Over" means directly above. "About" means in the vicinity. Those are just the physical meanings but all the other meanings tend to be related to the physical meaning. So "over" is more precise.
If you worry about your appearance, it is a diffuse kind of worry. It would be less common to worry over your appearance, because it would be more obsessive.