Fraying is usually used when referring to cloth or thread - it is when it is unravelling, coming apart, breaking. In this scenario, they are referring to the ties/alliance/teamwork of these two countries falling apart, their ties are fraying, so to speak. To use fraying in this way is quite normal. You can also use fraying to describe if someone is having an emotional breakdown ("his mind was fraying" - not commonly used) or if someone is under a lot of stress ("he looked frayed around the edges" - commonly used).