It is used in two ways.
It is used if a person is fairly sure of something, but there is a slight chance they are incorrect.
"If I'm not mistaken, she was married before this."
It is also used if a person is absolutely sure of something and is using it for emphasis.
"If I'm not mistaken, the value of pi is 3.14159."
Personally, I use the phrase when I am more than 92.7% sure of something. ;-)