The phrase "if anyone noticed him to begin with" is a peripheral thought that has no logical connection to the rest of the sentence. It could be turned into a separate sentence without any loss of meaning: "It was unclear where he stood in this conversation. Anyhow, nobody noticed him to begin with."

From the context you have posted, it means that the narrator isn't sure what Ken's opinion is altogether and that also he/she is not sure whether the people belonging to this seminar know it either: Ken doesn't seem to participate in the conversation, but is a mere observer to the chaos in the seminar. He may even be somewhat invisible to the other members of the class, and may have gone unnoticed for a period of time as he doesn't seem to mention any relationship with anybody in the seminar group.