Thanks a lot to all of you for coming to visit me. Thanks a lot, all of you, for coming to visit me. I would use commas in the second sentence because (all of you) is a parenthetical element that can be removed without altering the essential meaning of the sentence. I think both sentences are fine, when written correctly.

thanks a lot all of you for coming to visit. "to" is not necessary.