A "likeness" is a resemblance -- it's something that looks like, or resembles, something else. If you draw a portrait, you try to create a "likeness" of the person you are drawing. In this context, it seems that the thing the speaker sees isn't necessarily a creature. It strongly resembles a creature, but the speaker doesn't seem very sure about what it really is.
You're right that the phrase "as like to" is a fixed formula. It's literary and perhaps a bit archaic now. It means "as much like" or "as similar to." So if the creature-thing is "as like to a man as a circle is to a sphere," then it just as similar to a man as a circle is similar to a sphere. It probably doesn't specifically mean that the creature is one-dimensional like a circle, although it might mean that. Probably it means that the creature is like a man in most ways, but different in some particular, important way.