It means that your opinion is different from the speaker's. For example: A:"Playing with girls is boring" B:"Speak for yourself! I love playing with girls."

And sometimes a sarcasm, like "says you!", bad grammar though.