Almost right - you can't omit the article 'a' before 'waste of time'.
You can say either "It's a waste of time talking to you" or "Talking to you is a waste of time". The first sentence uses the dummy subject 'It', while the second uses the gerund phrase 'Talking to you' as the subject. The meaning is the same.
Many sentences of this type can be expressed in these two ways. For example, "Talking to you is difficult" or "It's difficult talking to you"/"It's difficult to talk to you".