* I'm a native speaker of American English and I've been hearing/using this phrase for a long time.
Wow, interesting question. To be "down with" someone or some idea, is to agree with it. To be "down for" an activity is to be enthusiastically interested in going along. To "get down" with someone means to dance with them, or to engage in something even more physical with them. To "throw down" means to get involved in a fist-fight.
I think that all of these senses of the word "down" contribute to it's general reception and interpretation. The implication perhaps is that there's maybe something illicit or secret going on. Its use is certainly always very informal.
As for etymology, it's a pretty old African-American vernacular idiom, and I'm not sure we could guess exactly where it came from. Maybe it's a gambling reference; to have money "down" on a team/boxer means that you've bet that they will win; I think the impressionistic sense of the phrase is that you've thrown in your lot with whatever person or idea that's being discussed.