As I understand, "子供の時、" is the thema of the sentence.
"子供の時に" isn't the thema, it's under a control of next verb. "に" is the mark of the verb-control.
Like " when i was ~
Like "when i went to ~
〜の時にの方がmore specific time だと思います。😊