From the Oxford Advanced Learner's Dictionary:
quack
noun
/kwæk/
1 the sound that a duck makes
2 (informal, disapproving) a person who dishonestly claims to have medical knowledge or skills
quack doctorsI've got a check-up with the quack (= the doctor) next week.
cloister
noun
/ˈklɔɪstər/
1 [countable, usually plural] a covered passage with arches around a square garden, usually forming part of a cathedral, convent or monastery
2 [singular] life in a convent or monasterythe calm of the cloister
It seems that the author meant people who pretend to be experts (=quacks) establish themselves in isolated groups (=cloisters).