Search papers, labs, and topics across Lattice.
This study investigates the efficiency of multi-agent document assessment in retrieval-augmented generation by analyzing the performance of 7B-9B instruction-tuned models across various QA benchmarks. The findings reveal that weaker models benefit primarily from per-document isolation, achieving performance comparable to full multi-agent assessment, while stronger models require a focus on scoring quality, which is addressed through a novel Reasoning-Score Coupling method. The proposed MADARA architecture leverages these insights to provide a lightweight, adaptive routing solution that generalizes effectively across different model families, significantly reducing computational costs.
Assessment-free isolation can match the performance of full multi-agent assessment, revealing a surprising efficiency in weaker models.
Multi-agent document assessment for retrieval-augmented generation is computationally expensive, driving practitioners toward smaller, deployable models whose assessment mechanisms remain poorly understood. We conduct a controlled study of training-free interventions on 7B-9B instruction-tuned models across diverse QA benchmarks, revealing a sharp dichotomy in how models benefit from assessment. For weaker baselines, the dominant mechanism is per-document isolation. Astoundingly, assessment-free isolation matches full multi-agent assessment, demonstrating that resolving multi-document context confusion, rather than scoring quality, drives outsized gains of up to 50 percentage points. Conversely, for strong baselines where scoring quality matters, we introduce Reasoning-Score Coupling, a label-free perturbation probe that classifies scoring behavior. Integrating these findings, we propose MADARA, a model-adaptive routing architecture. Crucially, MADARA's diagnostic thresholds derived from a single pilot model generalize zero-shot to four unseen model families, providing a robust, lightweight pipeline to eliminate computational overhead.
