I think that you thought of "connaissais" as the right tense to use, right? Wrong, and I'll explain why. The tense of "connaissais" is l'imparfait, meaning that the action of knowing began in the past and lasted for a while, but then, it stopped. But when you say "je le connais depuis toujours", it means that you know him for a long time. Clear? :)

I'm not a native speaker, but it seems right to me. 'Je le connais depuis toujours' would be translated 'I've always known him'. This French use of a present simple corresponds to the English present perfect.