The use of "came to realize" sounds more like British English than American English.
In American English the sentences would be:
I realized that I never really knew him at all.
I realized that I never really knew him.
Using "at all" adds emphasis to the feeling of "I did not know him" in the sentence.