"She heard the front door close behind him" - this is perfectly correct English. Using "closed" is a mistake, because the conjugation has already happened with "hear".

The first one is right. "close" not "closed"

I believe the second one is only right in this context: maybe in gossiping or something like that, somebody hearing about something someone else said or a rumour. Like: He thought he had said the door swung open, but SHE heard the door CLOSED behind him. "Heard" as in found out something out or being told something, rather than actually hearing the door itself closing. I think "heard that" would be better than just "heard" in this situation. If it is in a narrative context, so like in a story, I believe the first is right where you are literally saying she heard the door close.