Once upon a time, a group of people discovered the orange fruit and found it look like gold ("or") and taste like "angel," so they called it "orangel."
Before that, the color "orange" was but "red-yellow."
However, as people were always lazy, and as time went by, they were reluctant to write or pronounce the word correctly.
Gradually, the "l" in "orangel" was dropped and what's left was "orange."
So yes, the fruit before the color.
Just kidding, I made up everything except the "red-yellow" part and the sequence of things.
Orange (aka, China's apple) was indeed a name for fruit (derived from Sanskrit and had gone through a series of changes in a series of languages) before it became a name for color.