A girl can sometimes say this in order to be humble, or perhaps to indicate that her looks are not the most important thing to her.

Yeah scruffy to me always means a man who hasn't shaved. It's related to hair. I can give you two real world examples. Someone told me last week, that I look better "without that scruff on your face" and then somebody a few days ago said they "liked my scruff." In both cases they were referring to facial hair. When people look like they've just rolled out of bed we say they are looking "rough". Like "Steve is looking rough today." or we might say they are "a hot mess." The woman looks very neat. If you asked me to point out the scruffy one, I would say it's the man. Look at his scruffy beard. For the woman you could say she is dressed or looks comfy or casual.

She means she is not wearing formal clothing.