Informal usage; "men who specialize in mathematics" or in "budgets", or "accounting". Or who work with quantitative figures (numbers). Those who are not "numbers guys" are bad at using numbers, so they cannot read mathematical phrases well. Conclusion of specific quote: They are bad at math.

In a business situation, the "numbers guys" are the ones who are concerned about questions that have numbers for answers: For example - "What are your profits?" "How much will it cost?" "How many people do you need?" "How long will it take?" And they want a quantifiable answer to their questions. The accountants, financial officers, and engineers are "numbers guys." They were making their presentation to people who would want to know the big idea, but not the details.