Yes, they amount to the same thing. If you say 'I was brought up' you are also putting your parents (and whoever brought you up) into the picture, but the essential information is exactly the same.
By the way, I presume when you say 'pretty small town' you mean 'relatively small town', as opposed to a small town that is also attractive?