The example you used is correct. We can also use look back, in the same way, this is a phrasal verb and it means - think of the past. For example; Looking back I wish I had told her before anyone else. I hope this clears it up for you :)

For my part, look back at/on the past are both OK.

Totally agree with Rianne, but 'looking back on' might work better with a specific event. "When I look back on that day, I wish I had behaved differently." This implies your direct involvement with the past you are talking about.