You can say 'My father in fact sneaked..' . The basic meaning would be the same. We use the construction 'did + [base form] ' to emphasise the statement. This is quite a common construction. For example if you say: 'Thank you so much. I really did have a lovely day today.', it is much stronger and more sincere than just saying 'Thank you. I had a lovely day today'. The 'did + [base form] ' construction gives added emphasis.

In this case, "did" is the action. "In fact" is used to emphasize that it actually happened. So if "did" is missing, it doesn't make any sense to put "in fact" because there is no action. I hope you can understand.