As Randy said, I'd can mean both 'I had' or 'I would'.
In your sentence it can only be 'I would', because 'I had' doesn't make sense in such a sentence.
"I'd never drive that far by myself" = "I would never drive that far by myself"
If it was:
"I'd never driven that far by myself" then it means "I had never driven that far by myself".
Now you tell me, what are the I'd in these sentences?
"I'd never wanted to drive that far"
"I'd never have driven that far"
"I'd like some ice cream, please"
"I'd been there before"