All the sentences are correct but for different reasons.
The basic phrasal verb is "to settle down". It has no direct object.
You can "settle down (in order) to do something". Here, "to" is part of an infinitive of purpose. This is the same construction that would be used for any verb e.g. I moved house (in order) to be closer to my work.
In the construction, settle down + to + noun/ing, "to" is a preposition. Prepositions are always followed by nouns, noun phrases or an -ing (gerund).