The answer has nothing to do with “verse”. It depends on which verb uses the verse. Hence you would “look at the verse”, “find something in the verse”, “attend a lecture on or about the verse”, or speak about the author of the verse”.

The most common of these is "in that verse." It refers to the content of one portion of a poem, song, or other text (like the Bible) divided into verses. For example, "In the third verse of the poem, the young woman expresses her joy at being home again. In that verse, the mood is positive." I can imagine a use for "at that verse," but it wouldn't be very common. If you are talking about a poem, song, etc., and you are trying to explain to someone which part of the text you want to discuss, you might say something like, "Starting at the third verse, please read until the end of the poem." or "Would someone in the class read this aloud? Please start at the third verse." In this case, you're not talking about the actual content of the text. You're using the verses as a kind of road map to tell someone where to look. At least in the U.S., it would be a little more common to use "with" rather than "at" in this context. I can't think of a use for "on that verse," but maybe someone else in this forum can. I hope that helps!