As an adverb, "carefully" would modify the verb "to be," essentially telling someone to exist in a careful manner. You could say "act carefully," though this sounds awkward. Because we don't want to modify the verb "to be" here, we don't use an adverb. Instead, the command "be careful" suggests that someone should assume a state of caution - of being careful.

Very nice explanation! Thanks a million!