The first definition you mention describes *a* town, which is somewhere between a rural area and a city in terms of size. The second corresponds more to the concept of *downtown”. In the cartoon, the man seems to be suggesting that they go to the new café in *their* town.

I think the meaning here could be either option, a small town or the city centre. It could mean a small town when someone is just outside in the country, or it could mean the city centre when someone is in the suburbs. In either case, it would mean somewhere more central with more activity.

The meaning of "town" is relative: it depends upon where you are. It has no constant meaning. Suppose you live in Manhattan, but away from the center. Then, going "to town" would mean going towards the center of Manhattan. Even though the place where you live has one of the highest population densities in the U.S., you would still not regard your home as being "in town". However, if you visited your friend across the river in New Jersey, you could refer to returning home as going back "to town". Thus your home is both "in town" and "not in town", depending upon where you are and how you are thinking. "Town" is also used as a term to denote an urban area where there is much activity. The expression "to hit the town" means to go into an urban place where there is much to do.