It's not more formal, but it is more extreme. 'Leave' is a neutral word. 'Abandon' is a more emotive word. For example, if a husband leaves his wife and family, it simply means that he goes away. But if he 'abandons' them, this suggests that he is heartless and selfish, leaving them penniless and having to fend for themselves.

I would say that "abandon" has a more negative connotation- for example 'an abandoned child' or 'an abandoned building' makes it sound sadder. To leave can just mean to depart from a place :"The train leaves Paddington for Manchester at 6pm" .

Abandoned suggests a stronger break with the object. E.g. the builders abandoned their work today ... (maybe they are never coming back, they gave up, they quit) Leave suggests they may come back. e.g. the builders left their work today... (they may come back tomorrow)