The two kinds of letters can be called "capital letters" and "small letters," or "upper case" and "lower case." Thus the "case" of a letter can mean which group it belongs to. The case of "a" is "lower case," the case of "A" is "upper case."
"Case sensitive" refers to how a computer matches or compares text. The comparison is "case sensitive" if upper- and lower cases matter, so "A" is different from "a." The comparison is "case insensitive" if "A" and "a" are considered to be the same. If you are comparing "District of Columbia" to "District Of Columbia", a case-insensitive comparison will say they are exactly the same, but a case-sensitive search will say they are different.
I have never heard the phrase "cap sensitive."
Notice that the phrase "case sensitive" always refers to a comparison between two pieces of text. It is NOT describing whether a piece of text is all upper case, all lower case, or "mixed case."