There is very little difference. The addition of "up" specifies that you used a telephone. However, the sentence already implies that. If you just said, "I called him," it could mean you called him on the phone, or it could mean you yelled "Hey you! Come here."
The verb "call" can be used in many ways, so we often add prepositions, turning it into a phrasal verb.
For example:
call up - call using a phone
call out - yell something, or point out someone's mistake
call to - yell to someone
call on - ask someone to do something, or visit someone
call off - cancel
call back - to ask for someone/something to return.
call in - report to by phone or radio, or calling to be on a live broadcast on TV or radio.
That's all I can think of at the moment, but with only a few exceptions, "call" can be used alone with the same meaning as these phrasal verbs.