Yes, you can say "seldom on time" Very good. Seldom means rarely or not often. I seldom arrive at work before 8:00 am. I seldom watch television. I prefer to read. Dogs seldom grow to more than 100 pounds.

"He/She is seldom on time." ...on time = punctual ...seldom on time = rarely punctual Seldom, as an adverb, has a "negative" meaning as it indicates a "low rate of occurrence/frequency". (It's rarely used as an adjective.) Examples: I seldom see her studying at the library. They seldom come here to visit me. Hope that helps. (^_^)/