I need to clarify what I said before. Consider the two sentences
#1 - I would be.
#2 - I would to be.
Both are correct but they do not have the same meaning. The other words in your sentences have nothing to do with the decision whether or not to say "to". #1 is by far the more common one, so I will focus only on the meaning of #2. In #2, "to be" is an infinitive behaving as an adverb modifying "would" and which serves to answer a question "why?" or "how?". For example, imagine this conversation, taken out of context:
person A: Would you?
person B: Yes, I would.
person A: Why would you?
person B: I would to be. (meaning I would in order to be)