'Taste' here means the ability of someone to find something enjoyable or not, inappropriate or not.
This was in the worst possible taste because it wasn't to his liking. This was in the worst possible taste because he found it offensive (or hurtful).
It probably originated from someone's ability to discern bad taste in food, or a flavor that was unpleasant. Then it became an expression to show a wider range of experiences, like someone's offensive behavior or words leaving a bad taste in your mouth, kind of like bad food would.
Then that behavior is 'in the worst possible taste'.
You can also say: 'poor taste'.