A wallet is a small folded case, expressly for carrying paper money, checks, and credit cards. It has very little room for any other object you might want to carry. Both men and women carry wallets, though men's wallets are usually smaller and a different shape than women's wallets.
A purse is a larger handbag. Typically, only women carry purses and a wallet goes inside the purse. In addition to the wallet, women keep a variety of other objects in their purses.
There is a special instance where a "purse" is a very small little pouch used to hold coins. In this case it is usually called a "coin purse" and coin purses are usually carried inside of a larger purse. Occasionally "purse" alone might be used to refer to a coin purse.