The first sentence is OK because "fast" is an adverb.
And the sentence example you wrote is incorrect, "We are early. I didn't expect us to get here so quick." because the adverb "quickly" should be used instead of the adjective "quick."
Examples of quick in a Sentence
They had a quick drink at the bar.
She took a quick shower.