In general, it could be either, or both. Actually, in that sentence, it probably means "often / frequently" (i.e. giving us background about the boys' relationship). On this occasion, it's likely that they didn't actually fight between the two pushes and the big boy's starting to cry.

I would understand that they fought often.

Hi Susanna, I agree with Phil on this interpretation. I believe the author here means that "in the past, these two boys had been in many fights" as in, fighting wouldn't be a new experience at this time. If I say "I read a lot" it can mean "I read often" or "I read many books" but it's an experience. So I think the interpretation of "They fought often" is good here.

lot pronoun a lot (also informal lots) ​a large number or amount ‘How many do you need?’ ‘A lot.’ Have some more cake. There's lots left. He has invited nearly a hundred people but a lot aren't able to come. a lot to do I have a lot to do today. She still has an awful lot (= a very large amount) to learn.