Yes. Tolkien moved the phrases around to keep a rhythmic pattern.

To long is to desire something very strongly. So he has a very strong desire (a yearning, a longing) to dance and sing. The issue with "by her" is its placement. If it was at the end, I would say that he wants to dance close to her, the way you thought. But the way it's written, it sounds as if he is now next to her, possibly sitting or being still, desiring to dance and sing.

HI l thank you for your mail you sent to me.please send me an email in ([email protected]) then i will send you my nice pictures.i love you. voice