You can't say "I've started...an hour ago" because "an hour ago" contradicts the tense of "I have". It's just like saying "I play the piano an hour ago" or "I will play the piano an hour ago" ie. it's logically impossible. If you want to say "an hour ago" in any kind of English sentence, you have to use past simple, past perfect (which is a combination of present perfect and past simple) or past continuous (which is a combination of past simple and present continuous). In this case, you just want to use past simple to say "I started playing the piano an hour ago".

A progressive tense describes an action taking place over a period of time. It should be pretty intuitive when you should and shouldn't use a progressive tense. Words like 'start', 'arrive' and 'finish' generally describe a point in time and you naturally wouldn't normally use a progressive tense, except when something is taking a prolonged amount of time. If played the piano from 7:00 to 8:00, you could say "I started playing the piano at 7:00 and finished at 8:00" or "I played/was playing the piano between seven and eight o'clock".

But can I say: I've started playing the piano an hour ago?

Thank you both for your answers! What other verbs are there, which you don't use with a progressive tense?

You can't use any perfect tense if you're saying when an action happened. You can say "I started an hour ago", but not "I have... an hour ago". You could also say “I have been doing... for an hour", but you'd have to state what action you were doing for the entirety of that hour. Saying "I have been starting for an hour" implies that you've been trying to start something for an hour but you've been procrastinating.