What you have written is possible, but it sounds better as "The impact as a whole would be considerable."
We normally use this expression to emphasize the idea 'cumulative' or 'complete.' Your example works, but because the expression modifies a noun, we usually see it immediately after the noun. Examples would be:
"It´s always sad to fire employees, but it's necessary for the company as a whole to improve."
"Prisons are often dangerous, but they make society as a whole a safer place."
If you have any other questions just reply or send me a message!