It's not formal, it's an idiom; however, there's nothing wrong in saying it (it isn't slang). It is often used at the end of a long statement to give a summary of what has just been said. So, for example, a financial officer in a company may make a long presentation showing that sales are down, profits are down, and salaries are increasing. At the end of this presentation, he might say, "So, in a nutshell, we need to start cutting costs as soon as possible!"