Search papers, labs, and topics across Lattice.
This paper introduces SEVA, a structured verification agent designed to enhance fact attribution in LLM-based systems by providing detailed evidence alignments, reasoning chains, and error diagnoses. By employing a novel process reward that decomposes verification quality into independent components, SEVA overcomes the limitations of traditional binary rewards, significantly improving both verification behavior and outcome metrics. The results reveal that SEVA not only matches the performance of advanced models like GPT-4o-mini but also generates richer, more auditable outputs, highlighting the importance of reward granularity in multi-component tasks.
SEVA reveals that a structured approach to verification can transform LLMs from generalists into benchmark specialists, boosting performance on specific tasks while enhancing auditability.
Hallucination is the reliability bottleneck for LLM-based agents, and fact attribution verifiers are the last line of defense -- yet today's verifiers emit only opaque binary labels, leaving agents unable to self-correct and operators unable to audit. We present SEVA, a structured verification agent that emits evidence alignments, step-by-step reasoning chains, calibrated confidence, and a six-category error diagnosis with actionable fixes. Training such an agent with RL is non-trivial: standard binary reward on multi-component output triggers advantage collapse -- within-group reward variance vanishes and the GRPO gradient disappears. We resolve this with a process reward that decomposes verification quality into five independent components weighted 70/30 toward process signals, restoring the gradient and inducing an implicit curriculum -- the agent first masters verification behavior (alignment 0.917 -> 0.997, format 72% -> 100%), then outcomes (F1 64.9 -> 69.0). Structured output further enables a Verify -> Reflect -> Probe -> Refine self-evolution loop, which over four rounds on a 7B model surfaces an unexpected structural finding: each round produces a benchmark-specialist, not a generalist (+15 pp on HaluEval, -10 to -14 pp on TruthfulQA in the same model, persistent at 4x data). On ClearFacts, SEVA-3B matches GPT-4o-mini (69.0 vs. 69.8 F1) while producing substantially richer, auditable output -- confirming a principle that should generalize: for any RL task with multi-component generation, reward granularity must match output granularity.
