Hi Shinhye.
I think it can mean more than one thing. It can be like what Steve's comment said. Or the first thing I thought about is that if we have bad memories or too many negative thoughts we hold onto, they can eventually effect our physical health any bodies. For example, if we do not let go of a lot our anger it can effect us physically in our hearts, blood pressure, etc.