"Pass" is an object moving. To pass from an individual means to give from one to another e.g. to throw "He passed the ball to his friend" or to give "Please pass me the vegetables". "Pass" can refer to a person or thing moved "He passed through town" = "He went through the town". It can also mean to die e.g. "He passed on" as in his soul moved from this world to the another.
Your confusion is understandable. "I'll pass on that decoration" = "The object will not come to me" similar to "It will not come to me". I didn't realize how weird a phrase it is until you pointed it out.