Search papers, labs, and topics across Lattice.
This study evaluates the effectiveness of various post-training methods鈥擥RPO, SFT, and DPO鈥攐n language models' ability to ground their responses in prompt evidence. The findings reveal that grounding improvements are primarily mediated by pre-existing mechanisms within the models, with GRPO showing minimal gains and DPO achieving near-optimal grounding performance. Notably, the research indicates that the enhancements in grounding are largely dependent on the model's initial architecture rather than the introduction of new components.
Grounding improvements in language models are largely driven by existing architecture rather than new mechanisms, challenging assumptions about post-training efficacy.
Language models can ignore prompt evidence when it conflicts with memorized knowledge. Post-training can make models follow such evidence more reliably, but it is unclear whether these gains require new machinery or strengthen machinery already present. We compare nine post-training arms spanning GRPO, SFT, and DPO from one starting checkpoint, with key comparisons extended across scales and families. We estimate a grounding direction from that checkpoint before training. Across five tested GRPO variants, grounding gains are small. For the two variants replicated across seeds, equivalence tests bound their effects below the conflict-SFT gain even as the rewarded metric improves. Conflict-SFT improves grounding moderately, while DPO drives grounding near ceiling on its matched distribution. Conflict-SFT and DPO largely use the same causal attention-head set as the starting model. Subtracting the starting-model direction suppresses both gains, while adding it to the starting model recovers 35% of DPO's gain at a dose passing all stated side-effect checks. After a supervised warm start makes the context answer appear in more rollouts, the same GRPO recipe adds essentially no further grounding gain. In our setting, grounding gains largely depend on machinery already present in the starting model.
