I wish it was/were a little warmer seems more logical

Laurence

The sentence as written is grammatically correct but sounds strange. An English native is more likely to use a positive statement such as "I wish it was warmer in this country. I'm freezing."  

A probable reason for this is the preference for the 'er' form - colder / warmer -  instead of 'more cold' and 'less cold' or 'more warm' and 'less warm'. Because of this, we change the sentence so that we can use the 'er' form.

I was it wasn’t as cold in this country. I’m freezing! 

Technically, you could say it the way you did, but it is not generally used and it doesn’t make as much sense.

The option in this exercise is (Cold)

I wish it was _______    ________   in this country. I'm freezing! (Cold)