“My patience has its limits. I’m not going to wait a month for an answer”.
“Being nice to people has its limits. Faced with a real enemy, you have to be tough”
“My English vocabulary has its limits. I can’t understand urban street slang whatsoever.”
Your statement doesn’t really say anything about what limits you face as a result of having a certain vocabulary. It’s not grammatically wrong but it doesn’t express a thought.