It means to set fire to something so that it is completely destroyed, so there is nothing remaining above the ground. You are saying "Burn this 4-metre-high building until it is 0 metres high."

To burn something to the ground means to burn something to the extent that there's nothing left ("to the ground" ie. the ruins are at ground level). For example, in the Iliad the Greeks burned the city of Troy to the ground.

This expression can be used in different ways however I believe the main meaning is to completely burn something or engulf in flames. That is my understanding.