Hmmm if you're speaking in general, "never give up" is correct — it's the sort of phrase you see on a motivational poster, something you might say in a general sense to inspire or encourage. For example, "you are so strong, never give up." Thinking back on it, this is something I see a lot on say like, fitness blogs, or study blogs, but I don't really use it in real life ever.
I would say "never give IT up" if I was referencing something specific that a specific person was doing. For example, I might tell a friend (or more likely, someone like my mom might tell me), "you're so good at piano—never give it up!" (it=piano) or "you're so talented at learning other languages—never give it up!" (it=learning other languages).
I think you already have the gist of it—"never give up" is general and "never give IT up" is only used if you're talking about a very specific thing that an individual is doing.
Hope this helped!