In this case the verb estar is an auxiliary verb, and together with a gerund give the sense of something being done and its duration.
a) yo estuve haciendo - preterite perfect
En aquel año estuve haciendo un curso de español. In that particular year, not anymore.
b) yo/él estaba haciendo - imperfect or continuous. (but it doesn't mean the action takes place in the present or near the present)
Hace tres años, mientras estaba haciendo un curso de español, conocí muchos amigo. This is one of many uses of the imperfect, to put two or more events in the past, on ocurred while the other happened.