They are both prepositions of place. Where is how you feel about yourself? It's in your hands. Of course, 'hands' is used metaphorically. 'In a nutshell' is a phrase that introduces a summary. The idea is that the speaker/writer will put the main idea into a small number of words so that they could fit in a nutshell. This is a metaphorical nutshell but, again, 'in' is just a preposition of place.