To drink tea is futre tense, "i want to drink tea" "i am planing to drink tea"
To be fond of something is to like doing something there for you use a doing verb.
I am fond of drinking coffee
I am fond of swimming
"I am fond of to drink tea" is incorrect gramma and strange sounding