I don't think 2 is unusual at all. The adverbs have different meaning or intensity. Very = a lot Quite= some/ a little So 'a very effective way' implies a high level of effectiveness while, 'quiet an effective way' implies it is a little effective or somewhat effective. They would both take the same placement in a sentence though.

Im curious what the incorrect word order was that the proofreader flagged. In the case of “very”, you have to put it between “a” and an adjective. (“A very good time”, “A very effective method”) But in the case of “quite “ you may put it either before or after the “a”. (“Quite a good time”, quite an effective method “) or (“A quite good time”, “A quite effective method“). The first is more common but the second is also correct. Before a noun, though, “quite” has to be used before “a”. (“We had quite a time!”, “He’s quite a guy.”) In this usage, by the way, “quite” is still an adverb. It is modifying an unstated adjective. And that adjective might be positive or negative. Strange!

I’ll attempt to explain this, sorry if it doesn’t make sense. In regards to #1, the word “effective” is describing the “way” so they kind of join together and become a thing in the sentence. A noun or thing (in this case the “way” must have either “a” or “an”, depending on whether the word after starts with a vowel or a consonant) so that’s why you got the correction to add “a very” although you could have simply added “an” but “a very” is much more convincing/persuasive. For #2, I think it will help to compare it to #1. In #1, the word “very” is describing the word “effective” (which is describing the “way”, so “very” and “effective” are both adding together to describe the “way”). Where as in #2, “quite” is describing “an effective way” as a whole (“quite” doesn’t join with “effective” to describe the “way” like in #1, but instead describes the overall “effective way”) I can’t think of any other adverbs that must be used like that but I think “quite” is just an exception. Apologies for making it so confusing, I hope it helps you a bit.