'Affair' is something that happened/ transpired, is happening, the general state of things, or sometimes it means a person's personal business. 'A sad state of affairs.' 'He needs to get his affairs in order.' A 'sad affair' would describe some event or something that transpired that was sad in nature. An 'affair' can also mean infidelity ( aka cheating), but that's not likely in this case.
'Sorrow filled' means something was sad: an event, a friendship/ relationship, a period of time, an experience, etc.
'You've both had your (fair) share of sadness' just means that both people have experienced sadness, sad events, troubles, etc.