Not in the US. "Petite" is a measurement for women's clothing sizes here. I think a petite blouse or dress is the smallest size. It's the french word for "small", as you probably know. In our legal system there is something called a 'petit jury'. It's probably from Latin and I don't know exactly what it means to be honest. Maybe it's for a small claims civil suit. So yeah, don't ever call a guy "petite" here. It might cause offense.

Petit is used with a masculine noun Petite is used with a feminine noun it means little