Usually, there's not a significant difference between the two. Your definition is quite accurate for both: violent anger, or extreme anger. With this meaning, "rage" or "fury" describe the emotion. To be clear, they can be used for situations that don't literally involve physical violence. EXAMPLE: He was overcome with rage/fury and screamed at the waiter. One subtle difference, is that fury is often used to describe the anger of God or a natural disaster (i.e. an "act of God"), hence the idiom "the fury of a thousand winds." EXAMPLES: The hurricane struck with such fury that the small beach town was destroyed. She was worried to face the fury of her god.