至少 is the right word used for 'at least'
最少 means 'no less than' and 'have the fewest quantity among '
I think in most of cases 至少 could be replaced with 最少 ,they are interchangeable in the examples below.
他的身高 至少/最少 180cm
我家到公司 至少/最少 6km
The example sentence to 最少 as the meaning of "no less than " :
我手上最少有500元(I have no less than 500 dollars in my hand)
The example sentence to 最少 as the meaning of 'have the fewest quantity among ' :
我擁有的錢是所有人之中最少的( I have fewest money among folks)
Hope it would help you to tell them apart.