1. Rather than continue the argument, he walked away. Fine. Emphasizes the choice to not continue the argument. 2. He walked away rather than continue the argument. Fine. Emphasizes the alternative action of walking away. 3. He walked away rather than continued the argument. Not correct. Here, "walked" anchors the action in the past and "rather than" needs to take the bare infinitive because it subordinating conjunction strips the verb down to its base form. 4. Rather than continuing the argument, he walked away. Grammatically correct but not really native. 5. He walked away rather than continuing the argument. Fine. Here, the emphasis is between the choice of the finite action (to walk away) vs. an ongoing action (continuing). It's an additional nuance beyond the first. However... "To continue an argument" *may* be more aligned with an "argument" meaning, a debate or an attempt of persuasion. In the noun form "an argument," feels possibly more balanced and two-sided if not within some sort of structure. More typical expression, maybe more rawly expressing inter-personal conflict and disagreement, the verb form is used--"to argue." Kinda the idea that two persons have an argument. Each, individual, chooses to argue on their own accord. More natural might be: Rather than argue, he walked away. (Emphasis of choice/decision.) He walked away rather than argue [[any] more]. (Emphasis of the action--walking away.) Rather than arguing [any [more]], he walked away. (Emphasis of decisive action to walk away rather than furthering the argument that was not leading to desirable or fruitful outcomes.) It takes two argue, but only one to not. :-)

All are correct, if you are relating the experience in the present or the past, or an ongoing action in the past or present. When did the action occur? Recently or in the far past?

Correct are 4 and 5 and yes they have the same meaning.