"Wala" is used to show possessiveness of a quality or trait. I don't know how you deduced that you can't use wala with "garmi."
"Aaj garmi wala din hay" is a perfectly fine sentence. It means that the day possesses the quality of heat or summer. "
"Voh peelay wala gamla hay" means that the pot possesses the quality of being yellow. The same is true for "kanton wala gamla."
The only sentence where you can't use "wala" is "yeh rangeen kapray han." And that's because the word "rangeen" by definition means something that is colorful or has colors. So it already shows possession of colors. So you can't use "wala," because it's already been stated that clothes have colors. However, to use wala you'll say "yeh rang wale kapre han."
Let me give you another example: "yeh kam wala kapra hay" means that this cloth has embroidery. However, you can also say "yeh kamdar kapra hay," this is an embroidered cloth. In the second sentence you can't use "wala," because "kamdar" already expresses the possession of the quality of being embroidered.