Coyness usually has a slightly negative connotation, in that it usually denotes false shyness, even a kind of flirtatiousness. Sometimes it is associated with a slightly irritating, or affected shyness in regard to sexual matters. So it can mean shy, but it is a particular kind of shyness, and the two words are mostly cannot be used interchangeably.

Coyness is also silent, or giving limited information about. A politician will be coy about the details of an election promise. Or a recent headline "Trump coy on China trade deal" - meaning Trump has given no details about the deal. When meeting new people it can be hard to tell if they are shy or coy. Maybe they are being coy, so you don't think they are an annoying/loud/heavy drinking/whatever person, or maybe they are just a quiet shy person.

I agree with Shirley that a display of coyness is often affected or false. It's not about shyness, though. If a person is shy, they may be introverted, uncomfortable in social situations or nervous about speaking up in public. Being shy is a personality trait - the opposite of being outgoing. Coyness is different - it's a reaction to a subject. I'd say it's closer to (feigned) embarrassment or modesty about something. Coyness is a reluctance to talk about a topic because you supposedly find it embarrassing. It could be about sexual matters, as Shirley suggests. Or it could be about your talents or achievements, if you pretend that you don't want attention drawn to yourself. Either way, it's irritating. Modesty is a good thing - coyness is not.