āClean upā is usually used to refer to messes you need to clean that will involve a lot of picking things up off the floor (or anywhere) and putting them away. To āclean outā is usually to clean something that holds other things (tub, box, desk, closet, etc.) and going through the container and throwing away some things inside you donāt need or just tidying it up.
If it helps, you can usually just use the word clean. āClean upā and āclean outā are just more specific ways to explain how youāre cleaning.