You are right : you wouldn't usually say 'so freezing' or 'very freezing'. This is because 'freezing' is an ungradable adjective.
Hot, warm, cool and cold, for example, are gradable adjectives, and you can use 'so', and 'very' before them.
The more extreme adjectives to describe temperature, such as freezing and boiling, are ungradable. You can say 'absolutely freezing' , but not 'so' or 'very' freezing.