You can just say "opposites attract" or if you want, you can also say: "like a moth to a flame."
I believe your use of "the wrong end of the stick" is correct. Basically this idiom means to have a misconception, so in this case, your question was misunderstood and Brian had the wrong end of the stick.
Great job with the idioms!