I when you arew the subject. Me when you are the object.

Take the "you and" out of the sentence. Then whatever works is what you should use when "you and" is in there too. I should go to work. --> You and I should go to work. He gave it to me. --> He gave it to you and me.