Yes, people talk about room temperature water frequently. However, not always in a good way. Like "oh gross, this water is room temperature." However, I did recall seeing a water cooler that had both a cold water and room-temperature water spout. It is also commonly used as a serving instruction in the world of food and beverages. "Serve at room temperature" for example. The only time it might be awkward is if the room is dramatically cold or something. Then someone might make a joke like "not THIS room's temperature I hope...hahaha". Another good expression mentioned here is lukewarn, however in my experience that means slightly warm. Either way it implies water that is not hot and not cold. It's like medium and medium well; in both instances you are receiving meat that is pink and not red, in contrast to well-done and rare.