Well, first (though it may be obvious), we must know that "hallelujah" is "a shout of joy, praise or gratitude".

This is NOT a religious song at all, even though it uses a lot of biblical phrases and allusions. It's actually very sexual, in some parts. The song is very poetic.. and can be interpreted to be about a man who is very deeply in love.

The entire last stanza which you posted:

Well there was a time when you let me know
What's really going on below
But now you never show that to me do you
But remember when I moved in you
And the holy dove was moving too
And every breath we drew was hallelujah


He is saying to his love that nowadays they don't get very "intimate", and that he is trying to remind her of the time when they WERE "intimate". The last sentence there beautifully dramaticizes what he remembers in his memory-- that while making love, every breath they breathed was hallelujah, was a shout of joy/praise/gratitude.

When he says "But remember when I moved in you", he is referring to when they were making love.

The "you" he is referring to is the woman he is in love with.

The lines "But love is not a victory march / It's a cold and it's a broken hallelujah" means that love isn't a victory, it's not something something to feel triumphant about, but that love can be very painful ('it's a cold and broken hallelujah).