It's a phrase used by musicians meaning "let's play some music", and usually involves improvising or making up new songs.

it's a suggestion to start something together (a party, playing music... etc)

There is another colloquial meaning-- it can mean to run off quickly, or to get out of somewhere quickly. so "let's jam" could mean "let's get out of here-- fast".