It can be correct using either "which" (= which floor out of all the floors in the building) or "what" (=not really thinking about the other floors in the building. Over the phone, I'd usually say, "which / what floor are you on?" On the elevator, perhaps the sentence fragment, "which floor?"