Search papers, labs, and topics across Lattice.
The paper introduces CounterRefine, an inference-time repair layer for retrieval-augmented question answering systems that addresses failures of commitment, where relevant evidence is retrieved but the system still produces an incorrect answer. CounterRefine generates a draft answer, retrieves supporting and conflicting evidence conditioned on this answer, and then uses a deterministic validation step to decide whether to keep or revise the answer. Experiments on SimpleQA show that CounterRefine improves a GPT-5 Baseline-RAG model by 5.8 points, achieving a 73.1% accuracy and surpassing the reported one-shot GPT-5.4 score by 40 points.
Instead of just gathering more context, turn retrieval into a mechanism for actively testing and refining a provisional answer, yielding substantial gains in factual QA accuracy.
In factual question answering, many errors are not failures of access but failures of commitment: the system retrieves relevant evidence, yet still settles on the wrong answer. We present CounterRefine, a lightweight inference-time repair layer for retrieval-grounded question answering. CounterRefine first produces a short answer from retrieved evidence, then gathers additional support and conflicting evidence with follow-up queries conditioned on that draft answer, and finally applies a restricted refinement step that outputs either KEEP or REVISE, with proposed revisions accepted only if they pass deterministic validation. In effect, CounterRefine turns retrieval into a mechanism for testing a provisional answer rather than merely collecting more context. On the full SimpleQA benchmark, CounterRefine improves a matched GPT-5 Baseline-RAG by 5.8 points and reaches a 73.1 percent correct rate, while exceeding the reported one-shot GPT-5.4 score by roughly 40 points. These findings suggest a simple but important direction for knowledgeable foundation models: beyond accessing evidence, they should also be able to use that evidence to reconsider and, when necessary, repair their own answers.
