'Yummy' is a childish and informal word used to describe something that tastes nice. It's a made-up word derived from the sound 'yum', which is a non-verbal expression of approval about tasty food or drink.
The word is used to and by children, and sometimes by adults - mainly young women. Older people use it less, and I have never heard a man use the word 'yummy'. It's sometimes used in advertising, especially for foods and drinks aimed at children and young women.
Of the three phrases you suggest, I've only heard 'Yummy in my tummy'. 'Tummy' is an informal and slightly childish word meaning stomach, and the two words are obviously put together for the rhyme.
And to answer your question, yes, it does sound childish. Unless you're actually trying to sound cute and baby-like, I wouldn't recommend saying 'Yummy in my tummy'. It isn't a normal thing for an adult to say.
You can use the word 'yummy', if you like, among your family and friends, but not in more formal situations. For example, it would be odd to use the word to a waiter or waitress in a restaurant - in this situation, you'd use a more neutral word such as lovely or delicious.