If "all" = "all things", it is more common to say "everything" and sounds better in both sentences. The first sentence applies in a specific situation. Perhaps the speaker had recently studied something and later says this sentence. The second sentence is a general statement. You could add the word "usually".

Both are perfect. They mean the same, but "feel" different. The first sentence states an experience you have. You could use it in a list of experiences: "I have gone to France, danced the cha-cha, eaten artichokes, and forgotten all I learned". This sentence says nothing about how many times you went to France or how many artichokes you may have eaten. I would say the sentence paints a picture of a person with many accomplishments who is eager to boast about those accomplishments. The second sentence is a simple statement of fact. It paints no picture at all. It feels less conversational, less chatty than the first. Talking to a friend, you are more likely to say the first sentence than the second one. You could list it in in a list of facts about yourself: "I went to France, danced the cha-cha, ate artichokes, and forgot all I learned". However, if someone said this to you, you would think that person was crazy. Why would he say such things? "I went to France" is just a fact, but "I have gone to France" sounds like something you might be proud of, an accomplishment.