It's a sports metaphor. In any game where a ball is used, it is essential to watch the ball at all times, to "keep your eye on the ball". For example, when you play golf, you watch the ball as you swing the club and hit the ball. If you do not watch the ball, if you "take your eye off the ball", you will certainly fail.