Search papers, labs, and topics across Lattice.
This paper introduces Self-Augmenting Retrieval for Diffusion Language Models (SARDI), a novel framework that leverages low-confidence tokens generated during the denoising process to enhance retrieval-augmented generation. By utilizing these lookahead signals, SARDI improves the retrieval of relevant evidence before finalizing outputs, significantly boosting performance in multi-hop question answering tasks. The results show that SARDI achieves up to 8 times higher throughput compared to existing training-free diffusion and autoregressive retrieval baselines, highlighting its efficiency and effectiveness.
Discarded low-confidence tokens during text generation can actually enhance retrieval performance, leading to a groundbreaking increase in throughput for multi-hop QA tasks.
Discrete diffusion language models generate text by iteratively denoising an entire response in parallel. At each step, they predict tentative tokens for every masked position, committing the confident predictions to the output and discarding the unconfident ones. We show that the discarded tokens are in fact a useful lookahead signal for retrieval-augmented generation: even low-confidence tokens often surface salient entities early in the denoising trajectory, enabling retrieval of stronger evidence before the output is finalized. We exploit this through Self-Augmenting Retrieval for Diffusion Language Models (SARDI), a dynamic RAG framework that uses these lookahead tokens to guide retrieval during denoising. SARDI is training-free, retriever-agnostic, and applicable to any reasoning-capable discrete diffusion language model. Across five multi-hop QA benchmarks, SARDI outperforms current training-free diffusion and autoregressive retrieval baselines at up to $8\times$ higher throughput.
