You can say "It was him" or "It was he". However, if you follow that with "who was holding something" then it is better to say "It was he". The reason is that "he" and "who was holding something" stand in apposition. That means they are parallel noun expressions that stand for the same thing. Since "he was holding", not "him was holding", "he" just sounds much better.
My choice would be.
"I don't remember whether it was he who was holding the thing, or if it was I."
(if you wonder why I said "thing", it is because there are many words that the iTalki filters don't like)