You're in Australia too, Katya.
Eldest can refer to people. My eldest brother, the eldest person in the group. It is not common to use 'eldest' for things, buildings, trees, etc. Elderly and 'elderly citizens' refer only to people, not things.
Oldest can refer to the age of a thing or a person. The oldest castle in the city, the oldest church in the town, etc. The oldest trees in the park.
Obviously the words have a common origin, but we use them slightly differently