I think David has a keen ear. Yes, the aspirated "h" does survive but it is fading.
I'm a U.S. native speaker and I grew up in the 1950s. I was brought up to believe that it was "better" pronunciation to aspirate the "h" in worlds like "when." However, it was vanishing even then.
It's the kind of thing that is subject to regional, age, and even family variation. I've been muttering "when, went, when, went, when, went" and I think that I do aspirate the "h" just a bit. Yes, I can feel just a hint of breath on the palm of my hand when I say "when" into it, and not when I say "went."
I remember being very bothered by the fact that it was spelled "when" but pronounced "hwen." What's particularly weird is that it was originally spelled "hw" in Old English!
As usual, Wikipedia has... more about this than I need to know. But it confirms that "The pronunciation of the digraph ⟨wh⟩ in English has varied with time, and can still vary today between different regions. According to the historical period and the accent of the speaker, it is most commonly realised as /w/, or in those dialects that retain traditional pronunciation, as the consonant cluster /hw/."
Wikipedia says that the spelling (but not the pronunciation) changed to "wh" in Middle English, but it doesn't say WHY. I think it's just about impossible to pronounce "wh" as spelled.