Basically they all mean the same thing, but there are shades of meaning between them. Example:
1. It was a sad affair between those two rivals.
2. It was a terrible criminal affair.
3. The company affair had been bad for business.
But
4. It was a horrible business.
5. It was a shady business.
6. The Chicago business came to an end.
7.. I will have nothing to do with this matter.
8. The difference between himself and his wife had now become a matter of common knowledge
9. However he decided to deal with this matter
or
10. The mechanism of a clock is a complicated affair.
11. They were flimsy affairs[ things], these cars.