One is a question, the other is you answering the question.
What would you be afraid to do?
I would be afraid of doing dangerous things like skydiving or swimming with sharks!
To be afraid to do implies the activity or thing may not be scary, but it worries you or scares you
To be afraid of doing implies you are scared of doing something