There's always someone who will try to discourage you.
In my experience, as an English speaker, I thought it was pretty simple to learn Spanish to a level that let me graduate from University, and lets me read somewhat.
Mandarin was more difficult as an English speaker as pronunciation was much more difficult, but grammar was relatively easy for an English speaker.
Korean pronunciation was easy, but grammar is much more difficult for an English speaker from that casual American background. It's difficult to compare straight across though as I've had to learn Korean for different levels and reasons than the other two.
There may be a fee similarities between Korean and Japanese, but I understand they do not share the same language family.
All that said, it sounds like you are good with languages, and you shouldn't be afraid to take on Korean if you want to. What will be difficult for you is going to be difficult to predict because it depends on so many things. It is doable, though. Have fun and good luck.