Track trials, findings, and clinical evidence as it lands.
Search papers, labs, and topics across Lattice.
This paper addresses the cold-start problem for prediction models when a new decision policy is deployed by leveraging counterfactual simulator rollouts. By training a forecaster on simulated trajectories under the new policy, the authors demonstrate improved zero-shot Sim2Real transfer compared to models trained solely on historical real data. Further refinement with early real observations yields additional error reduction, validating the utility of simulator data for cold-start forecasting.
Counterfactual simulator rollouts outperform historical data for cold-start forecasting under new policies, achieving up to 18.7% lower MAPE.
Deploying a new decision policy creates a cold-start problem for prediction models whose targets depend on the policy's actions: historical observations reflect earlier policies, while real observations under the new policy are not yet available. Simulation offers a way to address this gap by rolling out the target policy across counterfactual scenarios and using the resulting trajectories to learn how the system responds to those controls. The simulation-to-reality (Sim2Real) transfer of this simulator-trained model can then be backtested by evaluating it against real observations from past deployments. Using two real-world inventory-control deployments, we evaluate this process from three angles: simulator fidelity, zero-shot transfer to real behavior, and adaptation as real target-policy observations accumulate. The simulator-trained forecaster achieves lower point-estimate mean absolute percentage error (MAPE) than the same architecture trained on historical real data, reducing MAPE by 1.2-3.1 percentage points in Study 1 and 12.5-18.7 points in Study 2. After deployment, lightweight calibration using early real observations further reduces error by up to 2.5 percentage points. These results provide empirical evidence that simulator-generated counterfactual data can support cold-start forecasting under a new policy, and the resulting model can be further refined as real deployment data become available.