"My compliments" is a phrase we use to show appreciation for something. It's like saying, "Good job."
A common use is "My compliments to the chef," which is used when we enjoy a meal.
"Congratulations" is used when someone receives something good. Perhaps they won an award for their hard work or they found out that they will have a baby soon. When a person gets good news, we can often say, "Congratulations."