Topo morto gave you a good answer; [b] is voiced, whereas [p] is unvoiced. If the sound is voiced, the vocal chords vibrate, if it isn't, they don't.
These two ([b] and [p]) are a voiced pair. That means that your mouth is the same when you pronounce those two (it goes for any voiced pair). How your lips are positioned, your tongue. Here your tongue is low and relaxed, your lips completely close for a brief moment before opening which altogether gets you that [b] or [p] sound. The difference is in your larynx - when pronouncing a voiceless sound, it's relaxed and doesn't really move; when pronouncing a voiced sound, it's a bit tense and in a somewhat lower position.
Try listening them here:
[b]
http://upload.wikimedia.org/wikipedia/commons/2/2c/Voiced_bilabial_plosive.ogg
[p]
http://upload.wikimedia.org/wikipedia/commons/5/51/Voiceless_bilabial_plosive.ogg
Now, if you can't hear the difference at all, then I'm afraid that can't really be helped. But don't be discouraged right away, it could be that you simply haven't yet become aware of it, because you don't "expect" there to be a difference or something similar, something "psychological".
Another voiced pair is [s] and [z]. Mind that it's not the same as letters, as letter "s" can be pronounced in various ways in English, for example, words "salt", "measure", "busy". So, [s] as in "salt", "snake" and [z] as in "zebra" or "busy".
If you can hear the difference between those two, you should also be able to hear it between [b] and [p], and other voiced pairs (at least eventually, as I said).
Try listening someone pronouncing those words or pronounce them yourself. Do you hear the difference?
And one more thing, you mention aspiration. That something else, and both [b] and [p] can be aspirated.