They all mean pretty much the same thing, with very slight differences. "When I got home, my mom had made dinner." This just means exactly what it says: your mom had made dinner, and it was ready at the time when you arrived home. "Before I got home, my mom had made dinner." Here, there's a bit of emphasis on the fact that your mom had made dinner *before* you got home. You might say this if you were surprised that your mom had made dinner earlier than usual, if she normally doesn't make dinner until after you get home. "By the time I got home, my mom had made dinner." In this case, the emphasis is on "by the time," which suggests that you were running late, or that getting home took longer than usual. You might say this if dinner isn't usually ready when you get home, but today, it took you so long to get home that by the time you got there, dinner was ready.

You are missing some words. By the time/Before I got home, my mom had ALREADY made dinner. ALREADY can also go after 'dinner' ‘when I got home, my mom had ALREADY made dinner’? ALREADY can also go after 'dinner' This sounds natural. The way you have presented these sentences does not.

- By the time I got home, my mom had made dinner. - When I got home, I found my mom had already made dinner. - Before I got home, my mom made dinner. These three sentences all pretty much mean the same thing, with just small differences on emphasis and nuance.