An Exhibit - is something that is publicly displayed - ex: museums have different kinds of exhibits.
Supplement - Something added in order to 'complete' a thing.
Appendix and addendum are similar in that both can be short sections at the end of a book. However, they contain different kinds of information, and they are there for different reasons.
An appendix is a section of extra information that is useful to the reader. For example, a book on English grammar might have an appendix that lists the grammatical differences between American English and British English.
An addendum is a section of new material that is added after the first edition or first printing of a book. The author may be correcting something in the original book, updating the information in the book, or providing an explanation for the author's work. In any case, it's an afterthought. Legal contracts can also have addendums.