Strictly speaking a "schoolmaster" is a male teacher, but that's not the point that's being made here. The writer first says that the book has "a formal, serious style." "Schoolmasterly" is another way of saying that; it means "in the style of a teacher." There are cultural overtones but in the U.S. the ordinary word for a teacher is "teacher," and when I hear the word "schoolmaster" I think of somebody wearing a suit and conducting classes in a strict, formal, dignified, and slightly authoritarian way. "Schoolmasterly" here is similar in meaning to the word "pedantic," except that "pedantic" is a negative word, meaning somebody who is overly formal and overly attentive to unimportant details.

School-masterly is not a real English word. It is invented by the author to communicate the style of the book. Reading it is like listening to a school master (teacher). It is not wrong. The author is just having fun with the language. School master is an old fashioned way of saying school teacher. However, the principal teacher in a school is still called the Headmaster, or Headmistress for a female.

Traditionally it refers to men but it's in this case I feel like it could be generally referring to anyone who watches over a class.