It's: how many windows are there?
When you're asking a question you need to switch the verb and subject around. Egs:
- How many windows are there? There are five windows
- What time is it? It is one o'clock
- When are we going? We are going tomorrow
- Where are my keys? Your keys are on the table