Debris is more general and means anything that has been broken up into pieces.
Rubble is more specific and refers to pieces of a broken building.
It can also mean the loose, rough stones used as some kind of filler or bulk, used in masonry to build a wall. This is not often used.