To close out means to end. If I move to another country, I'd close out my account at the local bank. If a store is going out of business, it might have a closeout sale.
I can't think of a connection to a university, unless it means the class was full and no one else could register for it -- registration was closed out. But that's just a guess. Normally we'd say registration was closed.