Search papers, labs, and topics across Lattice.
This paper introduces LODESTAR, a novel method that utilizes reinforcement learning to optimize text interventions in retrieval-augmented question answering by scoring the uncertainty they induce in a frozen language model (LLM). By inserting a short fixed natural-language string into the prompt, LODESTAR effectively mitigates the issue of misleading passages causing the LLM to confidently produce incorrect answers, improving the mean answer F1 score from 0.5148 to 0.5339 across five QA benchmarks. The method outperforms fourteen existing configurations, achieving the highest exact match and judge scores, demonstrating its robustness in both in-domain and out-of-domain scenarios.
Misleading passages can lead LLMs to confidently wrong answers, but LODESTAR鈥檚 innovative polarizer intervention reduces this risk and boosts performance significantly.
Predictive-distribution entropy makes a strong selection rule in retrieval-augmented question answering: across five QA benchmarks, keeping the candidate answer that a frozen respondent LLM produces with the lowest answer-token entropy lifts mean answer $F_1$ from 0.4769 to 0.5148 over the retriever's top-ranked passage, with no gold answers. Yet this lowest-entropy rule, which prior entropy-based selectors adopt, fails in a specific and consequential way: a misleading passage makes the respondent confidently wrong, driving its entropy down precisely where the signal looks most trustworthy. We show that the failure comes from the passage the respondent reads -- and the context that passage is read in is an input we can intervene on. We introduce LODESTAR, to our knowledge the first method to score a text intervention by the uncertainty it induces in a third-party frozen respondent, compared across one question's candidates. LODESTAR uses reinforcement learning to train, once and offline, a polarizer -- a short fixed natural-language string inserted into the respondent's prompt and never into its weights; its training labels are built offline from gold answers and two LLM judges, and inference reads neither. Evaluating every competing selector under the same frozen respondent and the same candidate pools on 5,008 questions, LODESTAR attains the highest mean $F_1$ of any inference-ready selector (0.5148 to 0.5339), the highest exact match (0.4136), and the highest GPT-4o judge score of the frozen-respondent configurations judged (0.6435); its three-seed mean wins all 70 method-by-dataset $F_1$ cells against fourteen published configurations while remaining paired-significant against every one. The gain holds both in-domain and out-of-domain, and ablating the polarizer shows it is what makes the respondent read a misleading passage less often (26.0% against 30.3%).
