It is probably worth noting that while "No, I don't mind at all" is correct, typically native speakers will not respond so formally. Depending on where the person comes from you are likely to hear responses such as:
"It's fine"
"It's ok"
"Not a problem"
However, as janee mentioned if someone is busy the response will likely be negative.