You were close, but both sentences were slightly incorrect. You should say, "You are bringing people over who I’d rather have disappear from my life than stay at my home."
I hope this answer helps. If you are interested in taking English lessons, I am a professional English teacher on Italki and am currently accepting new students. Have a great day!