The breast here means the chest. When your chest heaves, it can be because you are out of breath after a long run. But in this case, it means that the person is breathing heavily in and out, with emotion. In fact, it's probably not literal but rather describes the tumult of emotions which are found within ("in the breast"). "but with" here means "only with / only in line with" "the deep" is a poetic word for the ocean. The heaving of the ocean refers to the up and down movements, the rhythm of big waves. So, if I understand it correctly, the poet is referring to someone who has a strong emotional attachment to the sea. The metaphor is that his or her emotional life is in sync with the movements of the waves. Nice verse! I'd not seen it before. Thanks for question.

"The deep" can mean "the sea." This use is literary and old-fashioned. Poetry is never easy to understand, and this is no exception. I'm not sure exactly what is meant in a literal sense. He is saying that somebody's breast, i.e. their chest, is not heaving, i.e. someone is calm. I can't tell if he is talking about his own "calm despair," or the body of his dead friend. The appearance of a dead body is sometimes interpreted in a positive way as "peaceful" and "calm." The whole poem talks about a calm landscape and a calm ocean. The image is that the ocean is calm as far as one can see, but farther out, in "the deep," it will be heaving. That much makes literal sense. When the wind does not stir up sharp, choppy waves, there are still long, slow, not-very-high waves far from land that are called "heave" or "swell." "The bounding main" also means the sea. "The main" means the sea and "bounding" means "with waves." I can't imagine how how farms and plains can "mingle" with it. Wait, yes, I can--it's the same image used in Arthur Hugh Clough's poem, "Say Not the Struggle." It probably refers to British coastal landscape somewhere, where the coastline is very tangled and the land is full of dozens of little tidal streams and estuaries connected to the sea. I'm not familiar enough with old-fashioned nautical terminology to know if "the deep" a specific meaning to sailors. I don't know if sailors use that word nowadays. It's probably a shortening of the phrase "the deep blue sea." Other similar words that can mean "the sea" are "the depths," "the abyss," and sometimes "blue-water." Other examples of use: "Many brave hearts are asleep in the deep, so beware! Beware!" --1897 song about people who have drowned. "'The Kraken Wakes' is an science fiction novel by John Wyndham, published in the United States in the same year under the title 'Out of the Deeps.'"

For anyone else answering: https://www.poetryfoundation.org/poems-and-poets/poems/detail/45333 In Memoriam A. H. H. OBIIT MDCCCXXXIII: 11 BY ALFRED, LORD TENNYSON Calm is the morn without a sound, Calm as to suit a calmer grief, And only thro' the faded leaf The chestnut pattering to the ground: Calm and deep peace on this high wold, And on these dews that drench the furze. And all the silvery gossamers That twinkle into green and gold: Calm and still light on yon great plain That sweeps with all its autumn bowers, And crowded farms and lessening towers, To mingle with the bounding main: Calm and deep peace in this wide air, These leaves that redden to the fall; And in my heart, if calm at all, If any calm, a calm despair: Calm on the seas, and silver sleep, And waves that sway themselves in rest, And dead calm in that noble breast Which heaves but with the heaving deep.

"Lost as a rabbit on Father's Day?" I enjoy idioms but have never heard this one.