I think that you thought of "connaissais" as the right tense to use, right? Wrong, and I'll explain why.
The tense of "connaissais" is l'imparfait, meaning that the action of knowing began in the past and lasted for a while, but then, it stopped. But when you say "je le connais depuis toujours", it means that you know him for a long time.
Clear? :)