#1 is correct
You can say #2 and people will know what you mean, but I think it is technically wrong. The problem with #2 is that it is not a complete thought. "Stealing the money" doesn't convey the fact that it was HE who stole the money. "Stealing the money is not a complete thought". Who stole the money? Did anyone steal any money?
If we add a few words to the end we can see that there are endless possibilities what the sentence can mean:
"He admitted stealing the money was not his own idea".
"He admitted stealing the money was a mistake".
"He admitted stealing the money was something he was not able to do."
So, you see, "stealing the money" is only part of a thought.
However, if you admit TO doing something, it is like a confession. It means that you did it.