I'm going to keep it simple. It can get complicated. In ordinary US English, "church," with a lower-case "c," can mean a building. It can also the congregation that meets to worship in the building. For example, "On the corner, there is a typical New England church, built of wood, painted white, with a tall steeple on it." "She's a member of the Grace Episcopal church, located on Western St." "Our church is holding a pancake breakfast Saturday morning. All are welcome to attend. Proceeds will benefit a homeless shelter." "The Church," with an upper-case C, means "all Christians." (Unfortunately, there is more than one large organized group that says "we are the true, original Christian church."). "The teaching of the Church is summarized in the Nicene Creed."

The Church is also often used to refer to the institution or specific type of (usually) Christian religion instead of a specific church location. For example, you could say that the Pope is the head of The Church or The Catholic Church.

If you say "to church", you mean to any church. It doesn't matter which church you go to. If you say "to THE church", you mean a specific church. You have probably already said something about which church you would like to go to.