Both sentences are fine. It doesn't necessarily mean you aren't smart, but that you lack common sense. You might have a lot of knowledge or education, but you are foolish in other ways. You could also say "You haven't got an ounce of sense."