I think you're looking for an explanation when there really isn't one. Just remember that infinitives go with certain (phrasal) verbs and gerunds with others.
As for whether the second word is an adverb or a preposition, just ask whether it actually works as one. I don't see either of them as a preposition.