"Sharp" is an expression to say that someone is smart, clever, witty. Usually you might hear "sharp as a tack," so I think the "sharp as a needle" is probably from an older writing.

There is an old expression that isn't used as much anymore which describes a person as having a "sharp wit." This is the same as saying someone is "smart" or "intelligent." There is, however, another possible meaning which is that not only was Jack's answer intelligent, but it was also unexpected by the man to whom he was talking. It says Jack's answer was "sharp as a needle" and needles always hold the danger of pricking you while using them to sew and often when you least expect it. The man mocked Jack, believing that he would not know the answer, but Jack surprised him with the right answer.

'Sharp' here means quick. It means that he answered immediately. It could also mean that it was a clever answer.