It's a reference to the literal practice of hitting someone in the knees with a bat, thus incapacitating them. As far as I know, this is/was practiced by the mafia or other such hitmen, cronies, thugs.
So the phrase can be used to indicate any dirty, cruel action used to eliminate an opponent, even if you don't mean it literally.
"Take out someone" or "take someone out" = eliminate that person/beat that person
"at the knees" = [hit them] in the knees