I would only use the phrase "all grew up" with the animal kingdom and with something added. For example: "My puppies all grew up to be huge guard dogs". Or "My children all grew up to be doctors." You can also use this phrase in the present simple passive to emphasise a progression during a child's development, e.g. "My son is all grown up. He just learned how to tie his shoelaces." Or, "My children are all grown up. They are all in college." Hope this helps.