To me, the difference would depend on what you are at/on the edge of.
'On' would be used for things that you can be on top of: 'On the edge of a cliff', 'On the edge of a balcony', or 'on the edge of the table'.
'At' would be used for things that you can't, or would not usually be 'on', such as 'at the edge of the lake', 'at the edge of the garden'.
Those are just general guidelines. In many cases you can use either. You might want to use 'on' to emphasise the fact that whatever is on the edge could fall off.