That is correct. If you were to follow "to" with a verb, it must be a gerund (meaning it ends in "-ing").
Examples:
"I look forward to talking with you."
"They look forward to playing soccer."
However, you can follow "to" with other parts of speech, particularly nouns.
Examples:
"I look forward to our talk." (In this context, "talk" is not a verb, but a noun.)
"They look forward to the soccer game."
Source: myself, American English