It just means different things suit different people. Each to their own. What Robbie Williams has tattooed across his clavicles.
No-one seems to really know what the "strokes" refer to. Affectionate gestures? Moving through water (ie. rowing/swimming)? Painting? The phrase seems less than a hundred years old, so if it has to do with whipping slaves, then the people who came up with the phrase certainly weren't being whipped.
I can't say that strokes definitely refers to drum beats, but there is a quote from Thoreau which gives the same sentiment:
"If a man does not keep pace with his companions, perhaps it is because he hears a different drummer. Let him step to the music which he hears, however measured or far away."
Interestingly, the theme song for the American sitcom "Diff'rent Strokes" makes reference to both expressions in the lyrics of the theme song.