Good news: apart from missing 'the' from 'the course' both your examples are correct, but with slightly different uses/meanings. MD's suggestion is also correct, but means something different from both of yours. The differences are very slight, so please be patient. You used the simple past tense for.the main verb, let us suppose you are asking about a conversation your friend had with a teacher yesterday. Did you tell him that you are.. implies that your friend still wants to stop taking the course. Did you tell him that you were.. implies that your friend wanted to stop yesterday; it doesn't imply whether or not they still do. If they have changed their mind, they might have to go back to the teacher to explain, so.the question could be an important one. MD's version could also use past or present, but the difference is not quite the same,: I will only explain it it you really want me to!

JImmy's answer is excellent. I just want to add; 'rules' about what tenses you have to use are not correct (as you can see from Jimmy's explanation). The different tenses are used in different situations. :)

have you told him that you are going to stop taking the classes/course ?