It's making you crazy because there are no consistent rules. But sometimes I point out to students other adjectives which go with the same preposition.
e.g. impressed, satisfied, pleased, disappointed - all come with "with" and have meanings related to personal judgment
It's something that one day I'd like to find teaching resources for. In the meantime, you could develop your own notes from your use of a dictionary and thesaurus.