The verb “look at” is a phrase and always needs “at” before the object. In your example, “down” is used to show the direction — apparently “he” is in a lower position than the speaker. Here, “down” has its literal meaning.
On the other hand, we could saying “looking down on him”, where “to look down on someone” means to regard someone as inferior (i.e., not as good).