Both are correct, the first is more casual and more likely to be said in American English.
In British English, especially if writing and wanting to use more formal English, you might find the first form used more like this: 'You can go WHENever you want', whereas you'd more likely write 'WHEREver you want TO'. The 'where' takes the 'to'.
But to be honest, no-one would look at you oddly if you used either in conversational English.