- A man behind the wheel of the green truck couldn't be my brother. He has been flying that time on his vacation.

or 

-A man behind the wheel of the green truck couldn't have been my brother. He was flying that time on his vacation. 

I would like to say that this man I saw that time could't be my brother, because he was in the plain that time. 

But I have to use this words like in the first sentence  - because it is translation from polish language