It means he has three brothers and is the fourth.

That's actually a very good question! I've never thought about it before, but there could be some ambiguity there, couldn't there? Often it is worded thus - born into a family OF three brothers - which means three in total. I feel the author, by using WITH, has introduced a lack of clarity. But I still assume the author means a total of three brothers. It's the 'family' with three brothers, not the man David with three brothers. I think! What do other people think? I'd like to know.

Yep, changed my mind... think it means 4 brothers!

Since it says "born into" I think it means he has three older brothers and he is the youngest boy. So there are four boys including David.