I'm not taking the written test but I'm taking the speaking test.

I would probably say "I can't take the written test, but I can take the oral." It depends a bit on who you are speaking to, but generally, this should be fine. Where is your teacher from? That can also make a difference in usage and how formal your language should be.

In America a spoken test is usually referred to as an Oral test, or oral presentation, It is more appropriate to ask your teacher, or make a statement of preference.

Thanks, Hassankan!

For you to be more polite to your teacher, just tell him or her, I AM A AFRAID, I CAN NOT TAKE THE WRITTEN TEST, I'D PREFER THE SPEAKING ONE.