Actually, it wouldn't a mistake at all if you do pronounce Й in russian words. If Russian speaks slowly enough you can hear Й. And the sound is reduced when you speak faster. So you can pronounce it without any doubt.
As for the "СВИДАНИЕ": yes, "В" is pronounced in this word. And usually, it sounds quite distinct.