In this context, 玩 means "stay at your home and kill some time." For example, watching tv, drinking tea, or simply sitting on the sofa.
Yes, it changes the meaning. If you left 玩 out, it's still correct, but the person won't know what you visit for.
玩就是“打发时间”的意思
如果省略【玩】,被问者就不清楚你访问他家是为了什么。