Costly

I don't think there really is one. The most natural thing to say, and something I've heard, is "that isn't cost-effective." In your example, "he does things in the least cost-effective way possible."

"He does things as expensive as possible" would be my choice. 'cost-ineffective' is quite good and does work in your example, but you have to be a little careful in it's usage. It's more an economics/business term or a more formal term for describing a business or economic practice (wasting money). You could say it in general conversation, but it's generally too formal.

I don't think there's an exact opposite but you could try something like "wasteful" or "inefficient", perhaps. Otherwise, as others have said, "it's not cost -effective" works well

Your sentence is grammatically correct but you’re most likely to hear something like “his (approach) is the least cost-effective “