The expression "on the other side of the fence" usually means that someone else thinks differently. They have a different view. "They are sitting on the fence" means someone is undecided. Using "fence" is not a natural expression when talking about going into another country. There might be a fence, but the important thing is the border line itself. If you are talking about someone else in another country, you could say they are across the border, but maybe that is more for places than for people. The town is just across the border. The large river is across the border. The word across is used more like an action, or a movement as well. To get to the town, or to the river, you need to go across the border. The person lives on the other side of the border. They live over the border. I believe this is more natural.