This is a very interesting question. I think I agree with Richard that "box of chalks" is correct in British English but it is definitely not correct in American English. I looked at Google Images and you can see a few boxes that say "chalks" on the front, but there are many many more that say "chalk."
My suggestion would be to say "box of chalk" since this is never wrong. If you are speaking to an American and say "box of chalks" they won't realize that this is actually correct in the UK and will assume you've made a mistake.
You can always make "chalk" plural by saying X pieces of chalk or X sticks of chalk. Of course, this is only true when you mean 粉笔 -- chalk has other meanings and uses (e.g. chalk in powder form) that are never countable.