A <em>penalty</em> is a punishment. <em>To penalize</em> someone is to give them a penalty. (A common collocation is "impose a penalty.")
<em>Penalty</em> is a general word. There are many kinds of penalty.
In some places, the penalty for some crimes is death.
In the US, my bank sells a two-year CD which pays a high interest rate if you keep it for two years, but if you want to withdraw your money early, you pay a penalty equal to six months' interest.
In a hockey game, if a player breaks a rule, he and his team might be penalized by putting him for five minutes in the penalty box; for five minutes, his team has to play without him.
A <em>ban</em> is simply a prohibition on something. <em>To ban</em> something is to impose a ban.
For example, the town I live in has just banned the sale of flavored tobacco products in most stores.
Eugene O'Neill's play, <em>Strange Interlude</em>, was banned in Boston. The Boston authorities would not allow it to be performed.
A common kind of ban is to prohibit a person, or a thing, from coming into a place. For example, most restaurants ban dogs--you can't bring a dog into the restaurant. To prevent movie piracy, movie theaters often ban cameras.
Online, when a person in a forum breaks the rules or becomes a problem, a common response is to "ban" the person, meaning blocking them from logging in.
A ban <em>can be</em> a penalty. For example, in an online forum, a serious rulebreaker might be penalized by being banned. Online, "banned" usually means blocked forever. A less serious penalty might be a one-week suspension, instead of a permanent ban.