'Give' is an extremely basic and general word.
'Supply' is more specific. If A supplies B with something, you know that:
- B needs this product or service
- there is an agreement between A and B about this, usually involving payment
- A supplies B repeatedly, at regular intervals, or whenever the supplies are needed