Yes, it makes perfect sense! The phrase 'ordinary but not mediocre' implies that something can be typical yet still valuable. It suggests that even if something is common, it doesn't mean it lacks quality. Native speakers might use other expressions like 'hidden gem' to convey a similar idea.
For example: Imagine finding a tiny bookstore in town. It looks normal at first, but inside, you find a special place with rare books and comfy spots to read. This store is like a hidden gem, giving you a special and cozy experience even though it might seem plain from the outside.