It is used when you want to describe something or explain something, but you cannot be exact. It introduces some degree of uncertainty into it.
He is kind of smart = He is a little bit smart, as you suggested.
It means that some things he does show that he is smart, but there is some doubt because other things he does are not very smart.
It is an informal phrase used in everyday speech, but I would not use it in any form of formal writing.
Other examples:
He kind of sneaked up on us
I'm getting kind of tired, let's start again tomorrow