As in spoken English and in a private informal communication you can say what you feel like :) It is not common to say 'sweet dream' though , "Sweet dreams" is the commonly used expression . Scientifically an average person has between 3 to 5 dreams a night ,if not more ! That is probably why it is not accurate to say ' have a sweet dream ' ! Otherwise in a flirtatious way you might even say " Have a sweet dream of me " :)