I would say yes. However... "Take it easy on her/him" implies not to be too critical. "Take it easy with her/him" might mean to go slow and not put a lot of pressure on a person.