The verb is simply 'rush' . If you rush someone, you want them to hurry up and do something more quickly than they want to do it. For example, a husband and wife in a restaurant: What do you want to eat? I don't know. I haven't decided. Come on! Choose something. Don't rush me. I need some more time to look at the menu. So if someone rushes you off the phone, it means that they want you to finish talking and get off the phone as soon as possible.

This means why are you being so quick, don't you want to speak to me? Rush off means to be quick.