yes, you can change it to that - it is grammatically correct, but very awkward. the original sentence is also awkward. the only time you would ever use the second version is for a test or grammar exercise. colloquially we would say , "this is the baby (that) she has to look after." often the 'that' is not spoken.