When we say something is tricky, we mean it's a little difficult or unusual in how it works. We don't usually say that about people. "You are tricky to have done this to me." sounds to me like, "You are sneaky to have done this to me." but it depends on what you want to say.