If you say 'must,' it means that the person certainly 'thought that from the start.' The verb 'should have' may be more appropriate to express regret here. i.e. Person A: "I wish I didn't eat so much ice cream! Now I have a stomach ache." Person B: "You should have thought about that from the start."

Marcia's right, you should use 'should have' instead of 'must have.' We use should when we are giving advice, which is basically what you are doing here, even though it is about regrets.