"Could you speak a little louder?" is better.

A common phrase - in the UK, at least - is: Can you speak up a bit, please?

It's a good question to remind me loud is adj and adv as well.

The first one is grammatically correct. The 2nd one should be "Could you speak louder". In this form, it is less polite.

They are both correct, but native speakers will say the first. “Could you speak a little louder”.