If you're talking in general, it's more common to use -ing, but you can use the infinitive as well.
If you're referring to a particular occasion, it's more common to use the infinitive.
The same is true with "prefer" and other verbs:
1) "I love traveling ("traveLLing" on my side of the ocean). What about you?"
"I prefer playing chess".
2) "I'd love to travel next summer, I haven't done it for years. What about you?"
"I'd prefer to spend my vacation ("holidays" on my side of the ocean) at home.
However, this is a subtlety, the difference is not very important.