So "never mind (something"), means forget about it , stop thinking about it, cast it from your mind. Other examples.
"Oh, never mind that, you sound be more worried about..."
Mind being used to mean think about is used in another idiomatic way, usually in a passive-agressive way if someone does something without thinking how it will affect you:
"Don't mind Me!"