Search papers, labs, and topics across Lattice.
The paper introduces QueryBandits, a model-agnostic contextual bandit framework designed to mitigate hallucinations in LLMs by adaptively selecting optimal query-rewrite strategies. It addresses the gap in hallucination mitigation research for closed-source models by using an empirically validated reward function to guide the bandit's learning process. Results across 16 QA scenarios show that QueryBandits, particularly with Thompson Sampling, significantly outperforms No-Rewrite baselines and static query-rewriting policies, demonstrating the effectiveness of online learning in mitigating hallucinations without model retraining.
Stop relying on static query rewrites for hallucination mitigation: QueryBandits shows that adaptively selecting rewrites based on semantic features slashes hallucination rates in closed-source LLMs, beating fixed strategies by up to 60%.
Advanced reasoning capabilities in Large Language Models (LLMs) have led to more frequent hallucinations; yet most mitigation work focuses on open-source models for post-hoc detection and parameter editing. The dearth of studies focusing on hallucinations in closed-source models is especially concerning, as they constitute the vast majority of models in institutional deployments. We introduce QueryBandits, a model-agnostic contextual bandit framework that adaptively learns online to select the optimal query-rewrite strategy by leveraging an empirically validated and calibrated reward function. Across 16 QA scenarios, our top QueryBandit (Thompson Sampling) achieves an 87.5% win rate over a No-Rewrite baseline and outperforms zero-shot static policies (e.g., Paraphrase or Expand) by 42.6% and 60.3%, respectively. Moreover, all contextual bandits outperform vanilla bandits across all datasets, with higher feature variance coinciding with greater variance in arm selection. This substantiates our finding that there is no single rewrite policy optimal for all queries. We also discover that certain static policies incur higher cumulative regret than No-Rewrite, indicating that an inflexible query-rewriting policy can worsen hallucinations. Thus, learning an online policy over semantic features with QueryBandits can shift model behavior purely through forward-pass mechanisms, enabling its use with closed-source models and bypassing the need for retraining or gradient-based adaptation.
