This is an idiom, or expression, so it does not need to be taken literally. That said, think of it this way: Each "time" in this sentence is a particular point in time. "From time to time" measures an amount of time between these two points. But we didn't say when those points were! Because we don't know when the two points of time are, we don't know how long is between them. We do know, however, that there is more than one time being mentioned, so it didn't happen only once. So, "from time to time" means that it happens more than once, but the amount of time between each event is unknown. Not only is it unknown, but it's not even consistent. If I say that "I eat sushi from time to time," maybe I ate sushi twice last month, but three times in the month before that, and four times this month. The time between eating sushi is not regular, it's just "sometimes".

I think you encounter this phrase more in literature, than hear it in conversation these days. Additionally, I don't think it's so important to take idioms so literally all the time. IMHO....

From time to time = every now and then = occasionally. I know this phrase means you occasionally do sth >>correct. I play golf from time to time / every now and then / occasionally