Both can suggest the idea of a long wait or delay e.g.
When at last they found him he was half dead.
Or
After 3 attempts Alice finally passed her driving exam.
(eventually is used in same way)
Only finally can be used to introduce the last idea in a series e.g.
When you leave take a key, a bag, a coat and finally a scarf.
Both can be used as an exclamation.
At last! Where have you been?
Finally! What happened?
'In the end' is another expression you can use to show something happens after a lot of uncertainty.
Did they get married in the end?