The phrase is sort of idiomatic, but basically, the word "only" is modifying "fair" and saying that what you're doing is not something special, it is "only" (merely) an act of being "fair."
Brian has given a good example. A few more might help.
"Since all the other kids got candy during class, it's only fair that my son should get candy, too." (I'm not asking for a special treat for my son. I just want him to be treated like the other students.)
"I bought dinner last night. It's only fair that you pay for the movie tickets tonight." (I'm not asking you for a free gift. I just want us to both pay equally when we do things together.)
"You're the one who made this mess, so it's only fair for you to clean it up." (Asking you to clean up your own mess isn't a punishment. It's just fair treatment, since you're responsible for it.)