Hi Mohamad,
Accent trainer Phil explained this well to you in terms of numbers.
The situation where you have to be careful is where the word mean is actually used as not nice. So for example, he's mean to everyone.
In this case you can't substitute the word average as it would not make sense.
Average is also more used to describe situations or objects. This is where you can also have a difference in meaning. For example, the quality of this coffee is average (i.e. the coffee is ordinary). In Australia, we also have a slang use for the word mean among teenagers. This is where in fact it takes the opposite to not so nice. For example, where a teenager can say "her dress is really mean", this would imply the dress is amazing.