This is correct for #1 and #2 but not for #3 and #4. May or Might would be correct in sentences #3 and #4.
He may have gone out by now........meaning (maybe he has already gone)
He could of gone out by now,,,,,,,,,,,,meaning t(he could have gone out)
He would of gone out by now........meaning (normally he has already gone out or has already left by this time..... so he may not be there)
Saying "I could have saved some money" would refer to a situation that has already occurred. Example: I bought these clothes at Macy's but if I had bought them at Walmart I could have saved some money.
You could say "By the end of this year I could have saved some money if I had not lost my job". In this case you are refering to what may of happened in the future (saving money) if something had not happened in the past (losing your job)
This would work with "I could have.....I may have.....I might have....I would have...
Hope I didn't confuse you with my explanation.