You might get different answers from different regions for this question. Literally, they mean the same thing, and all three can be used either literally or metaphorically. I'll add the following notes, though, based on my own experience:
"Let go of it" is most often used literally. If two children are fighting over a toy, the parents might tell them both to "let go of it," meaning that they should stop grabbing the toy.
"Let go" is the same, and is probably the more common form. If children are fighting over a toy, you can simply tell them, "Let go." ("Let go of it" is a little more emphatic.)
"Let it go" is more often used metaphorically, in reference to something like a grudge, a regret, etc. If you complain all the time about how your ex-girlfriend left you, someone might tell you, "The relationship is over, so just let it go." The meaning is that you shouldn't keep thinking about it. You should mentally/emotionally "let go" of it.