These two sentences are absolutely grammatically correct. Their usage depends on the level of formality you want to express. My being here is so inappropriate. - it is more formal That's not appropriate for me to be here at all. - it is semi-formal

"It's not at all appropriate for me to be here." The reason "that" doesn't work is that it doesn't refer to anything. When you say "that's not appropriate" the question that comes immediately to my mind is "What's not appropriate?". "It" doesn't suffer from the same problem. "It" refers to "for me to be here at all". I can see why this might confuse you. The difference is subtle. "That" requires more definition than "it". "That" needs to be made more explicit. The listener needs to be told what "that" refers to. "It" is more forgiving to the speaker.

They're very similar. You should just say "It's not appropriate for me to be here at all." Instead of "That's not appropriate for me to be here at all."