They are both correct. Which one you use would depend on what sounds nicer to your ears. Whichever you use, I would omit "of" and put an apostrophe : He fell for her charm(s)

Thank you.

"Charms" does not just mean beauty. See macmillan dictionary: "charms [PLURAL] pleasant or attractive qualities or features. Visitors are delighted by the village’s more rustic charms. The place held no charms for me – it was bare and isolated." https://www.macmillandictionary.com/us/dictionary/american/charm_1

Thank you, Chris.

A quick look at the Oxford Learner's Dictionaries has all the information you need. Meaning 1 - charm (personality). Meaning 2 - charms (beauty).