"I was there last summer" is the correct answer. We introduce an experience using present perfect, them move to past simple to give more details. We only use present perfect if there is a background to the story.

The second sentence isn't wrong, but it means something different to what you're trying to say. "I have been to France" means that at any time before now, I was in France. "I had been last France" means that at any time before last summer, I was in France ie. last summer you could have said "I have been to France". It's much more natural to say "I had been in France BY last summer."

"I was there last summer" is the correct answer because you use the past simple with a past time expression "last summer"

Just as a side note...you could also have said: "Yes, I have been there. I went there last summer."