Search papers, labs, and topics across Lattice.
This paper introduces RAPS-DA, a regime-aware peer specialization framework designed to enhance retrieval-augmented generation (RAG) by addressing conflicts between retrieved context and model knowledge. By categorizing conflicts into three regimes鈥擥rounding, Arbitration, and Resistance鈥攁nd employing regime-specific peer specialists for targeted supervision, the method effectively disentangles learning signals and improves robustness. Experimental results demonstrate that RAPS-DA outperforms existing approaches across various conflict scenarios and benchmarks, achieving significant gains without requiring stronger teacher models or regime labels during deployment.
RAPS-DA reveals that targeted peer specialization can dramatically enhance RAG performance by effectively managing knowledge conflicts without increasing model complexity.
Retrieval-augmented generation (RAG) improves language models by grounding generation in external context. However, it can be fragile when the retrieved context conflicts with the model's parametric knowledge. Such conflicts span a reliability spectrum, ranging from reliable and partially reliable evidence to adversarial context. Existing remedies often handle such heterogeneous conflicts with regime-agnostic supervision, which can conflate incompatible learning signals across reliability regimes. To disentangle these signals, we propose RAPS-DA, a regime-aware peer specialization framework that addresses conflict at two complementary granularities. At the sample level, conflicts are divided into three regimes, including Grounding, Arbitration, and Resistance, with one same-scale peer specialist trained per regime from a shared base model. Each sample is then hard-routed to its regime-matched peer for on-policy reverse-KL supervision. At the token level, a dual-layer selector uses inter-teacher disagreement, student-teacher divergence, and student entropy to filter uninformative or unstable tokens, upweight confidently misaligned ones, and gradually focus supervision on high-conflict tokens as the student matures. Gains stem from specialization at a fixed model scale, not from a stronger teacher, and the peer specialists exist only during training, so the deployed student requires no regime labels or peer access. Experiments on five conflict scenarios and two out-of-distribution benchmarks show RAPS-DA surpasses all prompting, decoding, fine-tuning, RL, and single-teacher baselines.
