Yes, there is rule. I. If followed by a particle or copula(-da/desu), -"Nan" before consonant [t], [d], [n]. "Nan-no" "Nan-desu-ka" "Nan-de" "Nan-to" etc. In this case, when we want to emphasize that we are asking "WHAT," it's ok to pronounce as "Nani." -"Nani" for all other cases. "Nani-ga" "Nani-wo" "Nani-mo" etc. II. If followed by a counter(to ask "how many/much"), use "Nan". "Nan-nin" "Nan-kai" "Nan-dai" "Nan-mai" etc. III. If followed by a noun(to ask "what ....."), use "Nani." "Nani-iro(what color)" "Nani-go(what language)" IV. There are exceptions. "Nan-youbi(what day of the week)"

If it is followed by an object with a preposition, "nani" is used. If the sentence doesn't have an object, then "nan" is correct. Also, "what?" alone would be "nani?", which I think is because it's easier to say. なんですか？Nan desuka? なんでしょうか？Nan deshouka? なにが言いたいんですか？Nani ga iitain(o) desuka? なにを考えてますか？Nani wo kangaete(i)masuka? Note:　なんで　(nande) ＝　どうして (doushite)