You can use both verb-ing (a gerund) or infinity form of the verb in this case. So "I love staying with her." means the same as "I love to stay with her." The gerund "staying" can be treated like a noun. Example: I love food. (subject + verb + noun) OR I love eating. (subject + verb + noun). Most of the time, English speakers use gerunds and infinity (infinity mean tense-less or not bound by time) interchangeably. Depends on the context, "I love staying with her" can mean that you love staying with her and NOT someone else, while "I love to stay with her" can mean you love to stay with her and NOT love to do something else. I hope it helps.

"I loved staying with her" is fine.

i love staying with her