It doesn't exactly work. Remember that rule about using "will" when you make a decision at that moment? If you said it, it would sound odd at best and rude at worst. As if you've blown off the invite. The answer you need is what Gearoid gave: "I can't come to your house tonight because I'm having dinner with my family." You use the present continuous plus a future time to indicate an arrangement exists. You and your family both know about the dinner plans, and you've agreed to be there. "Will" is still just an idea in your head.

I can not go to your house tonight ,"go " is better than "come "I think ,

Nothing's wrong with it. A similar sentence would be "I can't..... because I'll be having dinner with my family. Your sentence states only that you will eat dinner with your family. The sentence I have suggested creates the feeling that the dinner with your family will take place over a period of time.

I can't come to your house tonight because I AM HAVING dinner with my family. Good work Greetings from Ireland Ger