Either one would be fine although "been to" is more clear. 'Gone to' could imply that you have gone there and possibly haven't returned yet. If you say 'have been to', it is clear that you are saying 'have you been there at some point in your life'

Forgive me, but you would never hear "I have been to camping" or "I have never gone to camping." You would hear "I have been to camp" or "I have gone to camp." The reason is that camping is used here as an activity, not as a concrete destination, as "camp" would be. The two forms are not quite the same. If you say "I've been camping.", you could be saying that you have camped and are continuing to camp now. If you say "I've gone camping.", you would be saying that "I went someplace and camped." In practice, people use either of the two forms when referring to something that happened in the past. With the first, you would need the context to know what time the speaker is talking about. [Curse all the English tenses! Not even WE can keep them straight!]