Search papers, labs, and topics across Lattice.
To extend reinforcement learning with verifiable rewards (RLVR) beyond formal domains like code and math, this work synthesizes objective verification for diagnostic reasoning by injecting known causal interventions into a controlled digital-advertising simulator (TRACE). This setup bypasses the need for costly human post-mortems while forcing agents to actively investigate confounded, multimodal telemetry using SQL and Python. Leveraging these synthetic rewards, RL fine-tuning elevates a Qwen3.5-35B model from 0.159 to 0.757 FullAttr@1, outperforming both prompted Claude Opus 5 (0.686) and a prompted 122B model while executing significantly fewer tool calls.
Model scale is no longer the sole bottleneck for complex causal analysis: pairing RL with simulator-synthesized interventions allows a 35B parameter model to surpass Claude Opus 5 on real-world diagnostic tasks.
Reinforcement learning with verifiable rewards (RLVR) has advanced language-model reasoning in domains such as mathematics and code, where objective answers are inexpensive to check. Diagnostic reasoning over complex data lacks this advantage: establishing the true cause of an anomaly often requires costly expert investigation and may remain ambiguous after the fact. We ask whether this asymmetry of verification can instead be engineered. We sample an intervention, inject it into a controlled simulator, and generate the observations it would produce. The hidden intervention provides an oracle label and objective reward, while the agent must still investigate noisy, confounded, and distributed evidence. We instantiate this approach in TRACE, a digital-advertising diagnostic environment with 12 root causes and fine-grained segment attribution. Agents investigate each episode using Python and SQL and must identify both the root cause and, when applicable, the affected segment assignment. On a held-out 235-episode test set, the strongest prompted baseline, Claude Opus 5, reaches 0.686 FullAttr@1. Supervised fine-tuning raises Qwen3.5-35B-A3B from 0.159 to 0.637, and subsequent RL with synthesized rewards reaches 0.757, outperforming all evaluated prompted baselines, including frontier closed-source models and a prompted Qwen3.5-122B-A10B model. The resulting policy also uses substantially fewer tool calls than the prompted 35B base. These results provide evidence that access to a scalable, objective training signal can be a more important constraint than model scale alone. More broadly, simulation-based verification can make otherwise ambiguous diagnostic reasoning tasks amenable to scalable reinforcement learning.
