It is a more acceptable way of saying "I'm f****d", which refers to the sexual act, literally, but which has devolved into the meaning that you wrote "messed up".
If you screw someone in a deal or exchange it means you somehow cheated, gypped, conned or duped them.