It could mean both. You have to look at the context. An escalationg fight might involve pushing, hitting etc, but in your sentences, I tend to believe there are just verbal exchanges. :)

In most cases referring to relationships, "fight" means an argument; using only words. However, the word "fight" can be completely dependent on context; if your second sentence followed a more serious idea like "Jake pushed his friend, and they ended up having a fight," then it would be more likely that it involved an actual physical confrontation.