I really like this question because it forces me to think. But I would say there really is no difference. I mean, you could say that "it's time we DID away..." is a little more intense. But I would say it makes little to no difference. "It's" is always "It is" or "it has" in sentences like "It's (it has) been a great day," "It's (it has) worked out alright". "It's" will never be "it was, " at least in American English.