I need all the people except you.
You can see that "you" can be in the set with "all the people", even if it is not in the context.
You are great, except for your laziness.
You can see that "laziness" cannot be in the set of any part of the previous clause. Even, it shows a contradiction.