The expression is "that a boy" but when said fast can sound like "attaboy". You would only say this to a boy in an encouraging or congratulatory way. For a girl, you would say, "that a girl". It is less common to say that to a man (that a boy) or a woman (that a girl) because it tends to be said from an older person to a younger person.

Hello, I support both Grace and Steve. One can use "atta-boy" and "atta-girl". They are both a very informal expression of encouragement. "Good job" would be more suitable for most contexts. Unfortunately, "atta-baby" is not colloquially used at this point in time. However, maybe you can change that! Take care.

Hi! You wouldn’t say atta-baby, but you could definitely say atta-girl :)

You would say Atta-girl :)