"What is the usage of something?" usually asks for a thing's purpose or for a how-to: "How do you use that?" would be an alternate way to say it.
"What is the use of something?" can mean the same thing but is often used in a rhetorical way that means that the thing has no use at all. Example:
- "We are lost in the desert but we have a sweater."
- "Well, what is the use of that?"