Physically - means really. The industry was moving location to a place further south than it was. Metaphorically - in this case means that its business was getting steadily worse.

Metaphorically, "heading south" means that things are not going well. For instance, if a company is "heading south," that probably means it's losing business or profit, and may soon need to close. Literally, "heading south" means physically moving toward the south. If you're traveling from Canada to Mexico, you're heading south. I don't know anything about the topic you're reading about, but the author seems to be saying that two changes were taking place in the textile industry at the same time: 1) things were not going well, and 2) business was moving to someplace further south. So the industry was "heading south" in two senses, both literally (moving toward the south) and metaphorically (losing profit, etc.)