This is about structure rather than meaning.
'To' is followed by the base form of the verb ( live ) when the 'to' is part of the infinitive. For example:
want + to live
try + to live
decide + to live
be able + to live
'To' is followed by the gerund (living) when the 'to' is part of the verb phrase. In these situations, 'to' is a regular preposition, and ( like 'with', 'for', 'at' and so on) it needs to be followed by a gerund. For example:
Look forward to + living
Be used to + living
Be accustomed to + living
A gerund is a verb in 'noun form'. So a good way to know whether to use the gerund or not is to say 'Can I substitute this verb for a noun?'. In these sentences, you could say, for example, 'I'm looking forward to your visit' or 'I'm used to a quiet life', so that's why we use a gerund. But you can't say 'I'm able to + noun', so that's why 'I'm able' has to be followed by an infinitive, not a gerund.
As you say, in most cases, 'to' + verb is an infinitive form. The use of the gerund form is more unusual, and restricted to certain expressions such as those above.