"... had touched her hand" if the contact was before the moment under discussion.
Example: "After the meeting, she told me not to touch her again. I didn't know that I had touched her hand."
"... was touching her hand" if the contact was at the moment under discussion.
Example: "During the meeting, she told me not to touch her again. I didn't know that I was touching her hand."
"... has touched" is incorrect because "I didn't know" refers to a past moment.