Use "regret to do" for things you will regret, and "regret doing" for things you have done.

As Peachey says, it depends on whether you are looking backwards or forwards. . If you did something, and then wish you hadn't, then you say that you 'regret doing' it. The phrase 'regret to' is both formal and formulaic. You see it in formal letters and official notices in set phrases such as 'We regret to inform you that..' Hope that's helpful.