If you're talking about your preference for one option over another option, the standard construction is 'prefer x to y'. In this phrase, 'to' is a normal preposition ( rather than part of an infinitive), so it needs to be followed either by a noun or a gerund, in this case 'being'.
For example, if you'd rather swim than sunbathe, you'd say "I'd prefer swimming to sunbathing". The speaker would rather have a partner than be alone, so he says "I'd prefer having a partner to being alone".
I hope that helps.