"Can" and "could" (and their opposites, "can't" and "couldn't") express either possibility (e.g., "We can go to the Chinese restaurant or the Indian restaurant") or else they express ability ("He can speak German"). In your example, you are expressing possibility. If the math contest is in the future, you'd say, "He can win the math prize" (meaning that he is good at math and it is possible for him to win). For the past, you would say, "He could have won the math prize last year" (but he didn't study/was sick/got too nervous, etc.). Again, you are expressing possibility, just in the past tense, so we say "could have" plus the past participle form of the verb. Don't confuse possibility with ability: if I say, "I could ride my bike for hours when I was a teenager," we are talking about youthful ability, not possibility (that would be, e.g., "I could have ridden my bike for hours, but I got a flat tire and had to go home").

kelvinho8, The problem with using “could” in the simple past with a specific event is-- how do you know if the sentence is referring to “ability” or “theoretical possibility”? He could win the Math prize last year. (wrong) …Mental ability or theoretical possibility? He was able to win the Math prize last year. (correct) Of course, you might ask the same question about “was able to”. He was able to win the Math prize last year. (correct)…Mental ability or possibility? Here is the explanation from A Comprehensive Grammar of English, the closest thing there is to a grammar “bible”. The construction “was able to” emphasizes not just the potentiality, but also the fulfillment of an action. For instance, “We were able to leave the camp early.” typically conveys the additional message: …and, moreover, we “did” leave the camp early'. Hence “was able to” but not “could” is acceptable in contexts implying fulfillment: I ran after the bus, and was able to catch it. (correct) I ran after the bus, and could catch it. (incorrect) I have taught this to my students, but in reality I don’t always follow this rule myself. There are times when “was able to” seems much too formal and I use “could” when speaking of an affirmative accomplished event in the past. Here is the context. Me: I won the dance contest yesterday. My friend: How is that possible? You can’t dance. Me: Well, I could YESTERDAY AT 10:OO P.M. My friend: You must have been drinking vodka again. By stressing YESTERDAY AT 10:OO P.M. I have subtly changed the meaning of the sentence to indicate that not only was it possible, I did actually win the prize then. Of course, in formal written English it is impossible to use intonation. Hannibal’s army could cross the Alps in 200 B.C. (incorrect)….Physical ability or possibility? Hannibal’s army was able to cross the Alps in 200 B.C. (correct)…This is the accepted correct usage you should follow in formal written English.

There is no "why". That is the way we speak. Period.

He could win the Math prize last year. is it really wrong?

The word "could" is often used as a form of verb. Sometimes used in a question, or to ask permission. It depends on the way a sentence is phrased when you use the word to express something in the past. What's the sentence you wanted to use?