When you use "I visited", you must tell us when. It's talking about the past. "I've been to..." means it happened sometime in your life experience. You don't indicate the time of the action when you use present perfect. It has nothing to do with sounding casual or formal. Both forms are standard, common English.

"Visited" may also suggest that your stay in the other country was relatively short and had a specific purpose..whereas "been to" is a bit more ambiguous and could have an indefinite period of time.

It is basically the same thing. But, if you want to sound more casual, you'd probably say, "I've been to."