"To take off" is American slang for "to leave" or "to go".
For example, "I'm going to take off" means "I'm going to leave".
While "Did he take off on you?" is correct, it sounds kind of awkward with the "on you" added to the end of it. "Did he take off?" sounds better.
Hope this helps. Good luck!