The phrase should be on 'alternate days' (plural), and yes, it means the same.

99% of the time, they are equivalent. In some cases you might want to say "on alternating days" instead of "every other day" so it's easier to understand.