A little tricky.
You are right in saying that "checking" here is acting as a noun. Simplifying the sentence gives us "Checking renders them less efficient at work." (As a side note, it should be "renders" with an "s" for the singular, since the act of checking is a singular noun.)
However, it is not just "checking" but *frequently* checking, and not just checking, but checking *cellphones* - so "frequently checking cellphones" becomes an entire "phrasal noun." (This is not the actual definition of a phrasal noun, but let's just use it for now to describe this phenomenon of a group of words acting together as one noun.)
In other words, *within* the phrase "frequently checking cellphones," "checking" acts a verb and therefore is modified by an adverb.
However, the entire phrase "frequently checking cellphones" acts here in this sentence as a noun.