"Go by" in this case means walking or passing by. For instance, if you're sitting near the window at a café, you might see people go by on the street outside. "Go by" doesn't ALWAYS mean this though. For instance, in the sentence "I go to work by train" it means they commute to work using a train, not that they pass a train on the way to work.