Kind of. It would be better to say "This area is off limits to anyone *WITHOUT* a special pass." OR "This area is of limits to *THOSE WITHOUT* a special pass."
The subject of the sentence is actually the 'special pass', so you can use 'without' perfectly fine as you don't actually need any sort of pronoun in it. That said, 'anyone' *is* a pronoun, though you can omit that too, with further rewriting (see below). Examples:
"If there's anyone without a waterproof coat, please make sure to borrow one, as tomorrow we're going rafting."
"You can't enter without a ticket."
"No entry without a special pass."/"This area is off limits unless a special pass is provided."
"No admittance without an accompanying adult for those under fifteen."
Hope that helps. :)