My shoes are worn out, without ever wearing them. I added are and a comma, which is correct. My shoes are worn out, even without wearing them for a day. This sentence is ok, but it might be a little bit confusing for some people. I changed with to without. I added are, for, them, and a comma.

My shoes wore out without my ever wearing them. My shoes have worn out and I didn’t even wear them for a day!

I agree with Samuel Harris.

I can’t believe that these cheap shoes are already worn out! I haven’t even worn them for a whole day and they are already falling apart.