Francesco has made a really good point. Sometimes, a factual and truthful statement can sound rude in English-speaking culture.
"I have to go" or even the informal "I('ve) gotta go" suggest that you don't want to go, but something else is causing you to go. You have an obligation somewhere else. These are perfectly fine.
You can also give your reason for leaving, instead of telling the other person that you are going. For example, "I have a class now", "Dinner's on the table", or "Someone's just arrived" are polite ways of excusing yourself and leaving. If you can learn to speak indirectly in English (but still make your message clear), then you have a very useful skill. :)