Search papers, labs, and topics across Lattice.
The paper introduces Adaptive Retrieval-Augmented Masked Diffusion (ARAM), a training-free framework that addresses retrieval-prior conflicts in retrieval-augmented masked diffusion models by dynamically adjusting the guidance scale based on the Signal-to-Noise Ratio (SNR) of the distributional shift induced by the retrieved context. ARAM strengthens guidance when retrieved context is reliable and suppresses it when noisy or non-supportive, improving the integration of external knowledge during the iterative denoising process. Experiments on knowledge-intensive QA benchmarks demonstrate that ARAM outperforms existing RAG baselines.
No training needed: ARAM dynamically adjusts retrieved context guidance in masked diffusion models based on signal quality, resolving retrieval-prior conflicts on the fly.
Retrieval-Augmented Generation (RAG) improves factual grounding by incorporating external knowledge into language model generation. However, when retrieved context is noisy, unreliable, or inconsistent with the model's parametric knowledge, it introduces retrieval-prior conflicts that can degrade generation quality. While this problem has been studied in autoregressive language models, it remains largely unexplored in diffusion-based language models, where the iterative denoising process introduces unique challenges for integrating retrieved context. In this work, we propose Adaptive Retrieval-Augmented Masked Diffusion (ARAM), a training-free adaptive guidance framework for Masked Diffusion Models (MDMs) in RAG settings. ARAM dynamically calibrates the guidance scale during denoising according to the Signal-to-Noise Ratio (SNR) of the distributional shift induced by retrieved context. Intuitively, the model strengthens guidance when the retrieved context provides reliable corrective evidence and suppresses it when the contextual signal is noisy or non-supportive. Extensive experiments on multiple knowledge-intensive QA benchmarks show that ARAM improves overall QA performance over competitive RAG baselines.
