Only "Why not do your homework?" is correct. No native speaker English speaker would ask "Why not doing homework?"

A Google search is not grammatically valid. All of the google results for "why not doing your homework" - all six of them! - have nothing to do with making a suggestion. You need a proper verb for making a suggestion: "Why not do your homework?" Pull off the question structures which make it a suggestion, and you have a clear imperative: "do your homework". Making it a negative question (Why not..?) results in a soft suggestion.

I agree with the other responses, you would never say "why not doing...". But I can't really think of anyone asking 'why not do your homework' because it sounds weird. There are much more common formations of the same question. For example, If you were asking why a specific person was not doing their homework you would ask them directly: 'Why are you not doing your homework', or 'why have you not done your homework' or 'why haven't you done your homework' or another, more verbose way would be "why would you not do your homework"