The phrase is sort of idiomatic, but basically, the word "only" is modifying "fair" and saying that what you're doing is not something special, it is "only" (merely) an act of being "fair." Brian has given a good example. A few more might help. "Since all the other kids got candy during class, it's only fair that my son should get candy, too." (I'm not asking for a special treat for my son. I just want him to be treated like the other students.) "I bought dinner last night. It's only fair that you pay for the movie tickets tonight." (I'm not asking you for a free gift. I just want us to both pay equally when we do things together.) "You're the one who made this mess, so it's only fair for you to clean it up." (Asking you to clean up your own mess isn't a punishment. It's just fair treatment, since you're responsible for it.)

Thanks for your example, Brain.

It may be easier to understand in context. Here is an example: "Last week, you gave me a ride to work while my car was broken down. This week your car is broken down, so it is only fair that I give you a ride in return for the ride you gave me." Essentially, we say 'its only fair' because of the reason that something is fair. Hopefully that helps in some way.