In general, with masculine nouns
-ы/-и is used for long words (with 2 or more vowels), and last vowel is stressed.
-а/-я is used for long words, and first vowel is stressed
Of course, there are many exclusions.
For short words, half of them have -ы/-и, half of them have -a/-я. Maybe there is any rule, but I haven't seen any.
And there are endings like -ья or so.