As Crazilchas says, only the first is correct as it stands, but I thought might help to explain why. Both sentences contain what are called relative clauses, which are structures giving more information about the noun phrase "first job". They are generally introduced in English for things with either the pronoun "that" or the pronoun "which". In your examples, the full expressions would be:
a) "He took the first job that he was offered."
b) "He took the first job that was offered."
Note firstly that they mean different things (in the second case it could have been offered to anyone, not specifically to him) and secondly, the pronoun in (a) is the object of the relative clause (to see this, compare to "he was offered the job"), and in (b) it is the subject (c.f. "the job was offered").
OK, getting to the point. There is a rule in English that you can omit the relative pronoun firstly if it introduces an "integrated" or "restrictive" relative clause (that is, one not set off by commas as "supplementary" relative clauses are) and secondly if it is an object pronoun. Now, both (a) and (b) pass the first test, but only (a) passes the second. That is why (b) is wrong if you omit "that" (or it could alternatively be "which").