Not sure entirely, but I assume it’s Because it is a mode of transportation. It just means to walk. The part of your body you are using is your foot. Feet is just plural and It’s implied both feet are used. People say by foot and on foot. It’s like when people say something is made by hand or handcrafted.

I think in Most languages there are expressions or short phrases that don’t translate literally and don’t need to be. Yes, we have 2 feet but we don’t need to actually say that. On foot is an accepted phrase. By is used with other forms of transport By plane By bus By car. Etc.

We see the afternoon/morning/evening as periods of time so we say In the morning for example. However, a rainy afternoon, the word rainy changes the emphasis and its better to say on. It’s a very small difference and hard to give a clearer explanation than that. Hope it helps. 🙏