In Russian there is a rule: when you don't know which letter must be written, you need to find a verification word.
This word has to be with similar root of the word and letter which you need to check, must be under the accent.
In your situation verification word can be "говор" (transl: speech). The accent is on the first "o" - "гОвор" and we can't say "гАвор", that is why we can't write "гАворить".
Word "говорить" you can pronunce as you wish, but write only "говорить".
But it is just one of a lot of rules :)