Yes. You can also switch those phrases and the sentence will still be correct.

If I'm not mistaken, I think it was raining yesterday.

It is a little repetitive, you could say "If I'm not mistaken, it was raining yesterday" because the I think is already implied. Either way sounds correct.