The verb "to shrug" means to raise your shoulders in a way that shows that you don't care.
"Shruggish" is making an adjective out of that verb,
although I don't think this is an actual word.
The sentence means "he listened in a way that showed he did not care."