A step-grandfather would be a grandfather related by marriage only. If your mother divorced your natural father, and re-married, that man would be your step-father. His father, in turn, would be your step-grandfather.
In practice, the word step- is used with mother, father, brother and sister, but very seldom with grandfather or grandmother. It is a technically correct term, however.