Like Jonathan said, they are often interchangeable but have some nuance. ‘In’ is more specific with regard to location--you’re taking the extra step to specify you're ‘inside’ of the place. In the context of Russian, 'at' is like 'y' and is like a genetive case usage. It can be thought of as 'at the place of X' or 'at the territory of X.' It could then be assumed that you're doing whatever normal activity happens there like shopping, eating, relaxing, seeing the doctor, etc. ‘In’ is like в/на and is prepositional case usage. So you could be ‘in John's house’ or you could be ‘at John's house.’ Or, simply, you could be ‘at John's.’ However, here you would not use the prepositional case ‘in’ unless you specify ‘house’ as it would sound wrong/funny. This can change when it is a proper name of a place or store–for example if ‘John's’ was the name of a restaurant, you could say ‘I’m in John's’ and would have the meaning of ‘in John's restaurant.’ Я в доме Джона versus я у Джона дома / я у Джона. I'm not a native speaker, but I understand these to be equivalent. Or with a doctor's office: ‘I am in the doctor's office,’ ‘I am at the doctor's office,’ or ‘‘I am at the doctor's’ all have the same meaning. ‘I am in the doctor's (office)’ where the ‘office’ is presumed also sort of works, but you would not want to say ‘I am in the doctor,’ as that would sound very funny.

Saying "at" is like pointing your finger. If you ask me "where is Bill" and I answer "at the hospital", it is the same as pointing my finger towards the hospital and saying "he's there". It's not specific. Bill could be in the parking lot. Bill could be across the street in a coffee shop. "In" is specific. When you are "in" something, it surrounds you. That's why we say a person is "in love" rather than "at love". When a person is in love, it surrounds him.

It's a topic that can't be easily explained. In the textbook I use with students most often, there are three units on the differences between in, on and at with regard to location. With most of your examples either one could work in most circumstances, but there's a clear difference in meaning with "in" or "at" the hospital. If you've been admitted as a patient, you're "in the hospital" in American English or "in hospital" (with no article) in British English.