You weren't to know usage in British English. = depending on the situation. 'that's ok it wasn't your fault, you could not have known. Stop worrying about it, there are no hard feelings.
It is a sort of cover all scenarios expression where someone cause something to happen and it wasn't their deliberate or careless fault, because they did not or were not told enough information, or were inexperienced not yet trained etc. You make an invited guest a meal and then they cannot eat it because of a deadly allergy you were not aware of, you feel awful, and the guest says "you weren't to know"
Example that has happened to me.
You delay and take your time closing a gate to an apparently empty field when all of a sudden there are hundreds of young bulls surrounding you and you can't close the gate because they are all pushing you in every direction. End result is the bulls escape onto a main road.
In this situation I wasn't weren't to know that the empty field was not empty.
You didn't have to know needs the full context to give accurate answer.
Young bulls can charge up from the bottom of a slop where they can't be seen at 30mph and be on top of you in a few seconds whilst you back is turned whilst taking your time walking back a few yards to close a gate. It all ended with no casualties human or bovine.