You are right, it does change but the things are a bit more complicated. Whenever it ends with 1, and not with 11 it's singular: один (1) мальчик, двадцать один (31) мальчик , сто восемьдесят один (181) мальчик. Whenever it ends with 2-4, and not with 12-14 it's formed like genitive case: два (2) мальчика, пятьдесят три (53) мальчика, триста тридцать четыре (334) мальчика. Everything else, ie whatever ends with 11-19, or 5-9 is regular plural: пятнадцать (15) мальчиков, сорок семь (47) мальчиков, сто двенадцать (112) мальчиков, триста семь (307) мальчиков. The rule of thumb is whenever spoken number ends with the word "один" you use singular, and since 11 (одиннадцать) doesn't, you don't use singular here (and you don't use it for any number that ends with 11 for the same reason). Whenever it ends with "два", "три" or "четыре" you use this weird "genitive case" plural (I don't know how it's really called). Since 12, 13, 14 (or any other number that ends with any of them) don't end with such word, you don't use the weird rule here.

♂ 1 (21, 31...101) мальчик — один мальчик 2 (3, 4, 22...24) мальчика — два (три, четыре) мальчика 5 (6 ... 10; 20, 25... 100) мальчиков - пять мальчиков 1 (21, 31...101) яблоко — одно яблоко 2 (3, 4, 23...24) яблока — два (три, четыре) яблока 5 (6 ... 10; 20, 25... 100) яблок - пять яблок ♀ 1 (21, 31...101) девочка — одна девочка 2 (3, 4, 22...24) девочки — два (три, четыре) девочки 5 (6 ... 10; 20, 25... 100) девочек - пять девочек ♂ 1 год — один год 2 (3, 4, 22, 23, 24, 102, 104) года — два года 5 (25, 20, 30, 10, 100) лет — пять лет

To put it simply, there are 3 endings. 1) Nom sg (i.e. the dictionary form) is used after 1 and all the numbers when you CAN HEAR the word "один": 1, 21, 31 ....1001 ่один дом 2) Gen sg will be used after 2, 3&4 (when you can hear these words "два", "три", "четыре"): 2, 3, 4, 22, 23, 24, 32 .... два дома 3) In all the other cases it's Gen pl. пять домов