The concept here is that of the notion of existence. English user use 'there is / are' to explain the existence of something. In both of your example, we would more readily use 'there' to determine existence. There is milk in the fridge. There are chairs in the room. Use Q words and inversion to from questions: How much milk is there in the fridge. How many chairs are there in the room.

Both can be correct but "is there" is more common. The other expression is likely to sound as if you think milk might be kept in the fridge or also kept somewhere else, and you wonder how much is in the fridge versus elsewhere. You can say "How much milk do you/we have?" if you're wondering if you need to buy milk today.

Both are good and I have no preference for either one. #2 sounds just a little bit more like a question since it contains the questioning phrase "is there?". Both pose a question, of course, but #2 does so more emphatically. Both are grammatical, but the structures are different, and I find it interesting to examine that difference: For #1: subject="milk", verb="is", and the rest consists of adverbial clauses modifying the verb. For #2: subject="there", verb="is", predicate complement="milk" and this difference simply reenforces the fact that #2 sounds more questioning since the core of #2 is "is there milk?", whereas the core of #2 is just "milk is".

Both are absolutely fine

Also “How many chairs are in the room?” or “How many chairs are there in the room?” ? 🙄