I am not used to going to ... [place].
“To be used to” is followed by a noun or a gerund which acts as a noun.
“I’m not used to the rain.”
“You’re not used to being in the rain.”
“She’s not used to it raining so much.”
However, when saying “[subject] used to,” it is followed by a verb - or rather, “[subject] used” is followed by an infinitive.
“I used to play in the rain a lot.”
“They used to come to my house every summer.”
“You used to be my friend.”
Hope this helps!