That's just a fixed expression and there is almost no reason why. 😉😊

Each language has its own logic.

because in English logic "with" shows a togetherness. "by" in this context doesn't make sense as it would mean you made her fall in love, but maybe you're not the person who she loves!

It's just a fixed prepositional phrase. They vary a lot between languages and there's rarely an explanation for 'why'; it's just something to memorise. 'In love' goes with 'with'. In French and Italian it goes with the equivalent of 'of'. Etc. Matt

If you are looking for logic in any language... don't waste your time learning a language. Your question is... not very smart.