We drifted. How did we drift? We drifted apart. Apart is modifying the verb "drift", so it is an adverb. Alternatively, you could think of "drift apart" as a phrasal verb, so that "apart" has no meaning by itself. "far" is modifying "apart". Adverbs modify verbs, adjectives, AND other adverbs, so "far" is an adverb.

I did. but I'm not sure.

Why not use a dictionary to find out?