By using the past simple tense, you emphasise the fact that something was done and completed in the past. For example, if you were a new teacher in a school, wanting to know which aspects of English grammar your students had covered with the previous teacher, you would ask "What did you do in the last 2 years?" The present perfect progressive/continuous tense, however, is used to refer to past actions which have present consequences, usually visible ones. It also marks the longer duration of the action So, the second sentence would be suitable if you were, let's say, a teacher, surprised with your students' poor level of English. "What have you been doing in the last 2 years?" would then convey your irritation/reproach/ unpleasant surprise due to the fact that students who have been learning English for so long haven't learnt anything.
Perhaps I made it too complicated. I hope this was helpful :o)