I feel "not just because A but more importantly because B" sounds a little odd.
If you start a phrase limiting the range of something with "not just", it should be followed by a phrase that supplies the rest of it, like "but (also)", rather than "more importantly" which is about depth than range.
If you must use "but more importantly", then instead of negating it with "not", leave it positive with a qualifier, as in:
I love this singer, first [in part / on the one hand] because her singing is good, but more importantly because her songs have helped me get through many difficult times.
This way, you first gives a partial or lesser reason and then follow it with the main point - you can't do it if you negate the first part with "not".