It would be grammatically correct, so of course it's possible. The issue is that that's not the way natives are used to hearing it expressed. I would recommend using the collocations that natives use. That will help others understand what you're saying.

The ‘s is often used with the barber because the barber is the person who cuts your hair and the barber’s shop is where you go to get your hair cut. So you’re actually going to the shop that belongs to the barber. The complete expression used to be I’m going to the barber’s shop, shortened to I’m going to the barbershop, shortened to I’m going to the barber’s, shortened to I’m going to the barber. Any of those is correct, though the barber’s shop is hard to say because of the double “s”.