Hi Book lover, ruggedly in this sentence means rough, or strong. Some men could be described as handsome even though they have delicate features. On the contrary, some men have rough masculine or (rugged) features which could make them handsome.