It loosely means "if he really meant what he said, it wasn't obvious"
A gibe (jibe) is generally a friendly mocking remark, but can sometimes be taken more personally. It depends mostly how well you know the person making the statement as to how you decide what the intent is