It's making references to words used in the Geordie (Newcastle) dialect.
In the Geordie dialect, 'wor' is a word which means 'our' (not 'hour.') It's pronounced the same way as 'war.'
So the joke is:
The guy says to the little Geordie soldier:
Listen, they've got war drums.
The Geordie boy hears it as: Listen, they've got wor drums.
In plain English: Listen, they've got our drums.
Hence, the response: the thieving bastards. The pun is that they don't have 'war drums', but 'wor' drums i.e. drums that do not belong to them and which they have stolen.
Stephen Fry didn't have a clue about it either, don't feel too bad.
By the way, Phill Jupitus says at the beginning of the clip, in a Newcastle accent:
Ferns (phones) make a canny noise. 'Canny' is a word which means great, fantastic etc in Newcastle. It's not niche knowledge, Fry should have known it. But he couldn't pick the accent and thought they were saying 'cunny'... which, well, is related to a pretty offensive word in English. Hence the levity.