A mill is usually a factory that makes cloth, or a place where grain is processed. A factory is where things are constructed (cars, turbines, etc.).

Brad has explained modern usage, though "factory" is the more general term and can be used these days to describe a"mill." Originally, in mills, the power was supplied by wind or water causing something to rotate or "mill". "Factories" used other power sources.