"error" is more formal than "mistake". People typically say, "I made a mistake", not "I made an error". In computer programming, there are "errors" in programs, not "mistakes". However, for the most part, the words are the same.
"wrong" is a state of being. People say things like "I was wrong". In general, this sounds much more humble than "I made a mistake".
"fault" is the responsibility for doing something wrong. For example, I might say that "It was my fault" because I was responsible for something and it failed to work.