Search papers, labs, and topics across Lattice.
This paper introduces two novel recall-based prompting strategies, Self-Recall (SR) and Question-Recall (QR), to enhance large language models' (LLMs) ability to operate under knowledge cutoffs. By addressing the limitations of direct-answer generation, the authors demonstrate that their methods significantly outperform traditional approaches, particularly in handling counterfactual questions. The introduction of the Multi-cutoff Historical Event Benchmark (MHEB) reveals that performance varies with cutoff distance, emphasizing the effectiveness of combining SR and QR for optimal results.
Recall-based prompting can dramatically improve LLM performance on knowledge cutoff tasks, especially for counterfactual queries.
Prompted knowledge cutoff instructs a large language model (LLM) to act as if information beyond a specified cutoff date were unavailable. However, prior work mainly relies on direct-answer generation, which struggles when post-cutoff knowledge is not explicitly queried but is only causally related to the question. To address this limitation, we propose two recall-based prompting strategies: Self-Recall (SR), which asks the model to restate its cutoff constraint, and Question-Recall (QR), which requires the model to recall question-relevant information valid under the cutoff. Across three existing benchmarks, our methods outperform both direct-answer prompting and conventional step-by-step reasoning baselines, with particularly strong improvements on counterfactual questions. To investigate robustness across different cutoff settings, we further construct the Multi-cutoff Historical Event Benchmark (MHEB), which evaluates the same question under multiple cutoff years. Results show that knowledge cutoff performance varies with cutoff distance, while combining SR and QR consistently yields the best performance.
