One big difference is the <em>sentence ending particles</em>.
For example,
(In a casual setting)
zo, for men, as in Okashii-zo. (Something's wrong.),
noyo, for women, as in Okashii-noyo.
Also, men tend to use words which originate from China, called 漢語(Kango) whereas women tend to use Wago(和語), Japanese before the addition of foreign elements.