It is a slightly profane way of saying "what on earth?", as in an exclamation of surprise. I personally have heard it quite often from people, native and non-native speakers alike, who like to avoid using 'very profane' words.

It's a byword for "Hell." Hell is sometimes considered to be a profane word, and isn't to be used lightly. Especially in the central part of the US, "Why the heck" was used as a substitute for "Why in the hell" or "Why the hell"