I've only heard "fly" when talking about trousers and pants. In fact, in Australia, I think it was the only word I used for this purpose when I was growing up. But we sometimes use "zipper" as well.
On the other hand, "zipper" can be used when talking about trousers, bags, pencil cases, tents, etc.