1. It costs more than it's worth - this is the best way to say this.
2. It costs more than it's value - the way this sentence is written right now doesn't make sense. If you use the contraction "it's", it means "it is", so the next word should be a verb.
- You could instead say, "It costs more than it's valued".
- You could also say, "it costs more than its value" (no apostrophe)
All in all, "worth" is the best word to use in this sentence. :)