Search papers, labs, and topics across Lattice.
The paper introduces Evidence-Diagnosed Intervention Training (EDIT), a two-phase framework designed to enhance the rubric fidelity of large language model (LLM) graders. By utilizing internal model signals to identify and revise problematic reasoning steps, and employing belief-guided reward shaping in reinforcement learning, EDIT significantly improves grading accuracy on real-world benchmarks. Experimental results show that EDIT outperforms traditional supervised fine-tuning and reinforcement learning approaches, highlighting the importance of internal diagnostics in achieving better grading outcomes.
EDIT reveals that targeted interventions based on internal model diagnostics can dramatically enhance LLM grading accuracy, outperforming conventional methods.
Reliable rubric grading requires more than accurate score prediction. Each judgement must be grounded in the mark scheme and evidence from the student answer. Existing credit-assignment and intervention methods, primarily designed for self-contained reasoning tasks such as mathematics reasoning, struggle in this setting because they do not identify where grading reasoning goes wrong or how the model's belief about the final mark changes during reasoning. We propose Evidence-Diagnosed Intervention Training (EDIT), a two-phase framework for training more rubric-faithful LLM graders. First, EDIT-SFT locates problematic reasoning steps using internal model signals: posterior belief over the final mark and input-grounding scores. It then revises only these local steps with help from a rubric checklist. Second, EDIT-RL calibrates the grader with belief-guided reward shaping, penalising large harmful belief drifts while still allowing helpful exploration. Experiments on two real-world, multi-subject grading benchmarks demonstrate that EDIT consistently outperforms strong supervised fine-tuning and reinforcement learning baselines on both in-domain and out-of-domain splits, with ablation studies confirming that internal-state diagnostics drive these gains.
