It’s definitely worth a visit.
Worth is used to put a price on something. For example, “the chair is worth $50.” In your example, this is the case because it’s really, “it’s definitely worth (the time and effort) of a visit” which is putting a price.
Worthy is only ever used as an adjective (1) or in the sense of “worthy of” (2), as exemplified in the following:
He is a worthy candidate for the post of President.(1)
He is worthy of recognition for his continued efforts.(2)
So if you wanted to you can say, “it’s definitely worthy OF a visit” but that’s overly formal and we wouldn’t say it like this in an every day context.