Notice that in Brian's examples you can interchange In fact and actually without changing much. So is their a difference? I am thinking that 'in fact" would be a slightly more assertive version of the two.
'In Fact" states "do not try to suggest otherwise because this is based in reality. To disagree would only make you wrong. Whatever you say, to be correct, you must start with this- as truth."
Actually is a more polite version of "this is true". It implies "I know you have likely thought differently, but this is what i know to be true- and I want you to know."
Just about the same.