It means a physical building. If you learn via distance learning, then you never go to a physical building, you learn on-line (or via some other method that does not involve going to a physical building).
Before distance learning existed, all learning was done in "bricks and mortar," schools, colleges, universities, houses etc
p.s bricks are the red, rectangular things that are used to make buildings and mortar is the kind of "paste," that exists between the bricks.