What you said was technically grammatically correct. However, there are some subtle things that would not be obvious to many non-native English speakers. In my opinion, the phrase "I was at church on Sunday" almost implies that there is a pattern of going to church on a regular basis. It also implies more of being part of an activity or a social gathering. But the term "I was at the church" has a tone that sounds more like the "church" is a building that they are referring to. If someone said this, then I would be less likely to assume that they were referring to church services. They might have been there for some other reason instead. However, the "Sunday" part does imply the strong possibility that they were there for church services because Sunday is the traditional day for these things. I'm not sure if this helped you or confused you, but those are my thoughts about this topic.