The meaning of those words depends a lot on the administrative system in the country you are asking about. You probably should specify which country you are asking about to get a more specific answer.
Here - boroughs used to be semi-autonomous parts of a city. They effectively don't exist any more, apart from historical references. A ward is an electoral area of a city - a councillor will typically be elected to represent a ward. We don't have districts here as such, although 'district' is often used in the name of a local body to show that it is a rural area, rather than an urban area.