In the U.S., yes. And it is a good expression to use. is a very natural, very common way to say it. It is factual; it is polite, it is neither a compliment nor an insult. Wikipedia has an interesting comment on various definitions of the exact age range, but 40-60 is roughly correct.
Of course, when I was sixty, I called myself "middle-aged" and my wife said "I guess you expect to live to be 120, then." :)