You are right. If you are talking about something attached in some way to a flat surface, whether horizontal or vertical, the usual preposition is 'on'. For example, 'There's a fly on the window' or 'There's a dirty mark on the window".
'In the window' is a little different. If we talk about something being 'in' a window, we are thinking of a window as being a display area. For example, goods are put on display in a 'shop window', which is essentially a glass cabinet. If you are using a window for display purposes, you can use 'in', even if what you are displaying - in this case a sign - is fixed to the surface of the glass.