You may use exactly, However if you are inviting friends for a 6:00 pm dinner and you say exactly, you are saying "Don't be late". I expect you to be here no later than 6.
It is grammatically correct, but a little strongly worded. If you are talking about when you eat, I eat at 6 every night is fine as well.