It is an expression coming from show business. It means that the earlier performance was a very good one making the audience expect more from you in comparison.
The expression can be used anytime a person who is very good at something is replaced. For example, if a respected CEO of a company retires, the person taking over the company might say that he is "a tough act to follow".