So the Licorice is Black.
As for the expression, "they looked DOWN ON" it means that the people think the
"other side of the family" to be of inferior quality.
The expression is often used this way. "We LOOK DOWN ON that kind of behavior.
It is also expressed like this. "We FROWN ON that kind of behavior.
or.... "I look down on that kind of people.
If we feel superior in some way, or think others show poor behavior or a lack of character, we might LOOK DOWN ON Them and refer to them as having "no class".
.