Many of the versions I've heard have been in accented English. Harry Belefonte's for example, is basically English.

I've heard it called Jamaican Creole English or Jamaican English, but more commonly I've heard it simply called Patois (pronounced as pat-wah).

As indicated by the title, "The Banana Boat Song", it's English. It is spoken to sound like it has a Jamaican accent.

I'm curious, what makes you think that it's not English? Or do you think it is and wanted confirmation?

I don't think it's "Jamaican creole." A "creole" is really a distinct language, which because of historical, social, and political biases is viewed as a corrupt or bad version of some other language. I would describe the Banana Boat Song as sung by Harry Belafonte as "English with a heavy Jamaican accent." My reason is for calling it "English with an accent" is that I can easily "translate" it into standard English _without any alteration of sentence structure or grammar._ "It's daylight, oh, it's daylight oh, The daylight comes and I want to go home... I work all night on a drink of rum [almost no changes needed]! The daylight comes and I want to go home. I stack bananas till the morning comes, The daylight comes and I want to go home... Come, Mr. tally man, and tally my bananas..." etc.