Technically, the two words should be pronounced differently /klooʊz/ and /klooʊðz/, however, many educated native speakers do indeed pronounce them the same. The "th" sounds are generally problematic for native speakers, and often get dropped in the presence of an s. Interestingly, "close," ( /kloʊs/) meaning "near," always sounds very different from the other two words you mentioned.
No, do not say "change your cloth." lol.