Tiger, good question.
Port is the left hand side, and starboard is the right hand side of a ship. It is not 'the port' and it is not 'the starboard', but you can say 'the port side' of the ship, and 'the starboard side of the ship'.
OK I got your question confused.
If you were to say "the ship is returning to the port' , then you understand that you are referring to a particular port,; probably the port/harbour you came from.
If you say 'the ship is returning to port' it could mean any port, to offload the crew members who are not behaving well.
Yes, it is much the same grammar as 'to go to school' and 'to go to prison/the shops/doctors' etc. It could mean any school/shop/doctor, etc, as a general idea I.e. any school, etc for education, any prison, any doctor for treatment, etc.