Your answer is fine. You can also say, "(I) hope you had a nice visit (with your family)."

The sentence is correct. To be more formally correct you should probably say "I hope...", but in informal conversation this is one of the unusual situations where you can omit the subject. Notice the use of the past tense 'had' - so this sentence would be correct if you were speaking to somebody who had already had their trip to visit their family. If you were wishing them well for a future trip then it would be, as Yi suggested "Hope you have.."

For everyday talking, your "sentence" is fine. I have "sentence" in quotation marks because, if you are using this in an English class for correct grammar, it will not pass as a complete sentence because the proper noun, I, is not included. So if this is a grammar class question, be sure to include "I".

It's bit Awkwark. It's better to say" hope you have a good holiday with your family

You're completely right! :)