In my opinion they would often be used in different contexts. To "reach new heights", tends to have a more grand and poetic meaning, often having to do with people and their hopes. "She always wanted to reach new heights." this could describe an ambitious person. To "reach a new high" is less poetic and more straightforward/literal. Perhaps a number on a graph has reached a new high. "The number of people supporting this idea has reached a new high". I'm sure there is some crossover, but in general this is how I think of it. Hope this helps!

reach a new high (................)? The team has reached a new high score. reach new heights = achieve or accomplish something new.