This, of course, still leaves you with the problem of how to understand and use these words.
Sometimes, the easiest thing is to check whether the speaker or writer is British or American - this will tell you whether the 'pants' someone is referring to are underwear or outerwear.
And which should you use? Well, in most cases I usually advise people to use the AE term if they're not sure who their audience is - for example, the American terms 'apartment' 'elevator' and 'movie' are generally better choices than 'flat', 'lift' and 'film' because they are unambiguous and universally used and understood
However, in this case, ''trousers' might be a better choice, simply because 'pants' may be misunderstood. If you say 'trousers', then at least this is unambiguous. It isn't a word which an American English speaker would necessarily use, but they would at least understand what you meant.