In the US, we only say ‘on the double’. My dictionary lists ‘at the double’ as British.

On the double. I’ve never heard anyone use the other one and I’ve never even seen it written.

Both are correct because they mean the same thing and can be used interchangeably. Both are phrasal verbs that mean 'Doing something very quickly'. I hope this helped

I checked and noted that both were ok. Personally I seem to hear "on the double" more often. Probably other teachers could confirm or give you more info.