If something or someone causes you to go the wrong way or get the wrong result, it throws you off.
The phrase comes from following something by its tracks and going off from the real path of what you are trying to follow because you saw false tracks. You were thrown off the trail.
"The extra wind threw my calculations off and our flight will be late."
"I was expecting 20 people, but the extra family you invited threw my plans off a bit."