In many cases, it will merely sound weird, not change the meaning. In other cases, such as your examples, it will completely change the meaning. 1. "She agreed talking to me" means that while she was talking to you, she agreed to something. (Here, "talking" would be understood as a participle.) Clearly, this is not the same as "she agreed to talk to me." 2. "I have finished to work" means that you have finished something in order to work (instead of whatever you were doing before). For example, "I have finished my studies, to work a paid job." Actually, it would be more frequent to use "quit" or "stop" in this way, for example, "she quit running to tie her shoe". We could remove the information implied by context, and simply say "she quit running" or "she quit to tie her shoe". Clearly, the meaning of the verb in the gerund is the opposite to the meaning of the verb in the infinitive. I didn’t invent English — I just teach it :)

This is a complicated topic. Simple Answer: It depends on the verb. You have the examples "agree to talk" and "finish working". You CANNOT say "agree talking" or "finish to work." This is decided by the verb. When you learn a verb, you have to memorize whether it takes a gerund or an infinitive.