You did something bad.
If I say,
It'd be better if you didn't do that. <- this doesn't make sense in that context, as it refers to the future.
I wish you hadn't done it. <- That's fine.
You just came in.
If I say,
It'd be better if you didn't come here. <- Again, that is future, so isn't appropriate.
I wish you hadn't come here. <- that's fine.