Seems really similar. Almost interchangeable.
However. . . . "Breaking up" seems more permanent.
"Breaking off" is more of a euphemism to soften the idea, since it is often harsh to admit to oneself that he is actually "breaking up."
Speaking of the situation in the past, we'd always say "broken up", and never "broken off". UNLESS, you'd want to say, "We broke it off + time period."