그 카페에 다녀요 is a little ambiguous. It has two meanings.
1) I go to the cafe frequently.
2) I work at the cafe./ I am on the staff of the cafe.
To avoid ambiguity for the case 1,
그 카페에 자주 가요. or 그 카페 단골이에요. (I am a regular at the cafe.)
is more commonly used.