Good link from Rong. This is the key section:
"The predicative complement here can only be an infinitival: All he did was ask a question. The bare infinitival is restricted to cases where the subject NP contains "do" in a relative clause, as in your example, but "to" can optionally be added: All he did was to ask a question. – BillJ Jan 4 '17 at 15:49"
In other words, when the sentence starts:
"All [subject] does/did is/was..." the verb form is without to or -ing e.g. complain, waste my time, ask a question"
In every other verb in the subject (noun) phrase, add "to" to the complement
e.g. "All he asked was to have a chance"