Could + Verb implies that at a certain time in the past someone was/wasn't able to do something. For example, Tom couldn't go, because he was ill. In other words, Tom was ill, and so he didn't go.
couldn't go = he wasn't able and therefore didn't go. He WAS sick, and he DID NOT go. 100% SURE! All past tenses. It all happened in the past. Ho connection with the present.
Couldn't have gone, on the other hand, contains a perfect tense (have gone). Well, I assume you already know about Perfect tenses, so I'm just gonna give you a descriptive example. Imagine you're Sherlock Holmes. And you're trying to find out if Tom went to the party or not. You argue that he was ill, so you think he couldn't have gone to the party (NOT SURE). Although, maybe he took some pills and was feeling better. He felt okay and nothing else probably could stop him. So he could have gone to the party (AGAIN NOT SURE).
Ultimately, it all comes down to facts (couldn't go) and opinions (couldn't have gone).