It would be more usual to say, "Why are you looking at me like that? Is something wrong?" or perhaps, "You're looking askance at me - is there a problem?"
Askance is not often used in conversation. It might be used to refer to a situation later: "Well, I could see he was looking askance at the idea, so I asked him what was wrong."