Literally, The word 숫자(數字) -- 수(數)의 자(字) -- refers to a digit, one of the written signs that present the numbers, and it is often used to show an amount or quantity.
The word 번호(番號) refers to a number used to show the position of something in an ordered set or list. So, when talking about a phone number, we say "전화 번호", not "전화 숫자". When counting the number, we should say that"숫자를 세다", not that "번호를 세다".