"Specialist software" sounds a little awkward. It might be interpreted as software used by specialists. "Specialized software" is correct, and I think this is what you meant--software that is specialized.
Otherwise, "Researchers have analyzed the results in detail using specialized software" and "Researchers have thoroughly analyzed the results using specialized software" mean the same thing, and both sound natural. I wouldn't say that either is more common than the other.
Side note: "analysed" is the British spelling and "analyzed" is the U.S. spelling. This is just a regional difference, not a grammatical point.