Everything you wrote is correct except: I am afraid of ghost. The correct way to say this is: I am afraid of GHOSTS (plural) or I am afraid of A GHOST.

I'm afraid to + verb > I afraid to go out / I am afraid to talk I'm afraid of + noun > I am afraid of ghost / I am afraid of going there alone I'm scared to + verb > I scared to go out / I am scared to find out that he is not my good friend

I am afraid to go downwards to the dangeons of the castle, because I'm afraid of ghosts. And if I had seen one, I would've been scared to death.