I would say "I bought the CD" or "I bought their CD" personally.

In colloquial speech, either of these could be said. However, it would be more common to say the second option, "I bought it on CD."