I think it is fine. If you mean "Yesterday, I was just thinking about you" or "I was just thinking about you yesterday!" then it's fine.