Little brother would be the natural way to say it. "Small brother" sounds like you're talking about his height. Big brother and elder brother are both natural sounding, but big would be used more casually. To use "elder" sounds a bit formal, like you're trying to be fancy or mature. But on the opposite side, "big brother" sounds a bit childish. The way most people would say this is a neutral manner would be "I have an older brother".