You can use either: Do you feel like going to dinner with me? or Do you feel like having dinner with me? The a is not necessary in the second sentence. The first one suggest that you want to go to a restaurant, whereas the second sentence is either eating in or out. Good luck with getting a positive response.

"Do you feel like having dinner with me?" is perfectly correct English. But I wouldn't say it. Nor would anyone else I know. The problem is the 'with me'. It's too...let's say 'personal'. It would make the other person feel uncomfortable. For example, if the other person had to say 'no' (perhaps they're not hungry, or they don't have time), they might think that you'd be offended. If you said this to a man, he might interpret it the wrong way, and think that you are suggesting more than having dinner together, if you get my meaning ;) It's more natural to say simply 'Do you feel like having dinner?' or 'Do you feel like going for dinner?' . You could also use one of the very natural phrases which Cerridwen gave you, 'Want to have dinner?' (which is short for 'Do you want to have dinner?') or the very common British phrase 'Do you fancy having dinner?'. You don't need to say 'with me' in any of these sentences. It is understood that you are suggesting that you and the other person have dinner together. 'Should we have dinner?' or even 'Let's have dinner' would work too as a suggestion. You could say any of these if the suggestion is for now, or you could add 'next week' or 'some time' for example, if it's for a later date.