It depends if the person is American or British.
British:
Trousers - The clothes worn on your legs (jeans for example)
Pants - A casual way of saying underwear. Specifically female underwear worn underneath trousers or a skirt.
American:
Trousers - This word is not used in America
Pants - The clothes worn on your legs (jeans for example). Known as trousers in the UK.