THE "I'm in charge of the finance department" = I organise and run the operations in this department. "I'm responsible for the finance department" = I must deal with whatever happens in this department. Essentially they mean the same thing, but "responsible" can also mean you are a manager higher up, ie. someone else below you is in charge, but what you deal with what happens.

They both convey the same meaning. You could also say: "I'm the head of the finance department." "I oversee the finance department." "I'm the boss of the finance department."