던 doesn't mean really "which". Even if it sounds like that, you shouldn't apply english to the korean language because sooner you'll get really confused with everything if you do ^^ 던 is a particle used when you want to make an adjective from a past-tensed verb. In your sentence 하다 is the verb, 했었 is the conjugated verb (resolute past), 던 is then placed at the end of the conjugated chunk to become adjective of "그 많은 추억들은" (all these memories) It has the same meaning with "the memories which we did together" but in Korean "we did together" is the adjective of the noun "memories" so "which" doesn't have a sense anymore. It is more like "the together-made memories" hope it helps.

The best way to think of -던 in English is as "used to". It means a state or action was stopped in the past, perhaps it was not finished. "The book I used to read", "The place I used to go", etc.