Yes of course! While babies are small, or little, compared to children and adults, there are big babies and little babies. Often someone will exclaim 'oh what a little baby', meaning a very small, newborn baby. You can also say 'tiny baby', meaning a very, very small newborn, perhaps a baby born early would be described this way.

See below.

Why、？

Sorry, but you can say it.

no you can't