You can rob someplace without anyone knowing that it happened. For example, you walk into the gas station and take a soda and cookies from the shelves without paying. You robbed the gas station or you stole from it, but you did not hold up the gas station.
If you hold up the gas station, not only did people know, but you used a weapon or the threat of a weapon to get people's attention and their money. The term comes from a robber with a gun who points the gun at someone and says "I have a gun, hold your hands up.". This is done to ensure they do not pull out a gun and it makes them defenseless.
Finally, to rob someplace is a general term and it includes a hold up which is a specific type of robbery.