In the context John Grisham use "adult conversation" he just refer to what adults, not children, usually talk.
“I wish. He’s too tired for sex. It’s not a priority anymore. And this is a man who could never get enough. I mean, we almost killed each other in law school. Now, once a week if I’m lucky. He comes home, eats if he has the energy and goes to bed. If I’m really lucky, he might talk to me for a few minutes before he passes out. I’m starved for adult conversation, Kay. I spend seven hours a day with eight-year-olds, and I crave words with more than three syllables. I try to explain this to him, and he’s snoring. Did you go through this with Lamar?”