It sounds like the story was telling about two travelers that had signs that said where they wanted to go. Perhaps they were hitch-hikers and wanted a ride some place. They might as well have written crude names for themselves because no one was going to give them a ride to any place, whether they were mean baby-kickers or nice men who just needed a ride. No one was offering them any help.

Maybe your sentence means that he does not believe anyone's words and think they are not worthy to believe

Maybe your sentence means that he does not believe anyone's words and think they are not worthy to believe

The answer is that people suspected that they were not worth their help.