No, you cannot use this expression except for a judge. This is because there is another expression using exactly the same words. "leave the bench" for athletes means that they get to play in the game, rather than sitting or standing on the side. "The bench" in this case is the seat where athletes wait while they are not playing. This expression is more common than a judge "leaving the bench". So, if you say "leave the bench" for anything else, most people would assume you mean that something had been idle and then performed some function.