Yes, you are right, different fruits have different 'things' inside them, depending on the size. If it's large and hard, we can call it a stone (GB) or a pit (US). You'd use these words for peaches, plums, cherries, dates and so on. If it's small, you'd call it a seed (mainly US) or a pip (mainly GB) . Apples, for example, have seeds/ pips.

A seed. As far as I can think, it's always called a seed. Apples, grapes etc all have seeds. Some fruit (collectively called "stone fruit") like peaches have a large hard thing called a 'pit' which has the seed inside so you might hear it called a pit for some fruit, but in general it's 'seed.'

If it's big, it's a stone. If it's small it's a seed, also called a pip.