Yes, an article is an article of clothing, specifically something you wear every day instead of something you only wear for a special occasion.

An "article" mean a "piece." So, an "article of clothing" is one item or one piece of clothing. A shoe, a shirt, pants.... It doesn't matter how often you wear it or whether it's casual or formal. It just means that you are talking about one item. In this case, the author is saying that politeness should not be saved for formal times or important situations, but should be used every day in common situations. "Articles of clothing" can also be used and this will distinguish that the speaker is talking about a group of clothes, but there is an emphasis on the individual items within the group.