You are right about the 'French Connection'. Inconsistencies in pronunciation often come from the various points in history at which these words were absorbed into the English language. The same goes for words which end in 'age'. The French words which have been part of the English language for many centuries acquired an unstressed 'English' pronunciation (like 'village'), while later imports retained their French pronunciation (like 'sabotage') , with some words (like 'garage') falling somewhere in between, with a somewhat undecided pronunciation.
As for the silent 'h', it's only 'heir', 'hour' and 'honour/honor' and its derivatives (honesty, honest, honorable etc) where the 'h' is never pronounced. In US English, 'herb' is sometimes pronounced 'erb', but not in GB English. There was a time when 'hotel' and 'historic' also had silent initial letters ('an hotel' and 'an historic' event' ), but this has almost entirely fallen out of use in modern-day English.