It is used to refer to something additional being done related to the first thing. For example, imagine that you work at a McDonalds. One of your tasks for the day would be to sweep the floors. Your boss might see you sweeping the floors and say, "can you wipe the counters while you are at it?". He is adding another related task for you to do since you're already cleaning (already "at it"). Another example: you could be going to the grocery store to buy cookies and your friend sees that. So your friend asks, "can you buy bread too while you're at it?" You're already buying cookies at the store so buying something else there is related and you can do it simultaneously.