It seems to me that the difference has to do with whether you're talking about a specific time or more generally.
"delayed by two weeks" means that some period of time has been extended by two weeks. For example:
"The setback has delayed the entire project by two weeks"
"delayed for two weeks" means that some event which was supposed to happen at a certain point in time will happen two weeks later instead. For example:
"I was supposed to go on a business trip tomorrow, but it was delayed for two weeks."