Yes. You can also switch those phrases and the sentence will still be correct.
If I'm not mistaken, I think it was raining yesterday.
It is a little repetitive, you could say "If I'm not mistaken, it was raining yesterday" because the I think is already implied. Either way sounds correct.