You are absolutely right about the existence of the difference between them.
~을 닮다 means to look after somebody/something.
So the sentence '나는 엄마를 닮았다' can be right, if I look after my mother.
But, strictly speaking, you cannot say '엄마는 나를 닮았다', because my mother does not look after me but vice versa. (since she gave me a birth.)
In the case of ~와 닮다, however, you can say both 나는 엄마와 닮았다 and 엄마는 나와 닮았다, because in this case it means basically I am simiiar to my mother and my mother is similar to me. Here, 닮다 means almost the same as 비슷하다, 유사하다.
But in its original intrinsic meaning, 닮다 is to look after, take after someone (or something). The meaning of resemblance in 닮다 comes from this basic idea.
In ordinary situations Korean does not much distinguish these different usages.
But when they have to distinguish it, they do distinguish it.
Of course, there are, in virtue of flexibility of language, some situations where it is not awkward to say 엄마는 나를 닮았다 or 넌 고양이를 닮았다. It all depends on the circumstances and context.