Search papers, labs, and topics across Lattice.
This study explores the application of large language models (LLMs) in forecasting tasks by analyzing their internal states with sparse autoencoders to differentiate between time-specific knowledge and generalizable patterns. The researchers found that enhancing time-awareness features significantly mitigated look-ahead bias in forecasting prompts while maintaining overall reasoning performance. This indicates that leveraging interpretable temporal features can effectively guide LLMs towards more historically grounded reasoning, improving their forecasting capabilities.
Amplifying time-awareness in LLMs can drastically reduce look-ahead bias, enhancing their forecasting accuracy without sacrificing general reasoning skills.
Successful forecasting involves identifying patterns between historical and future states of the world which generalize to future observations. We apply LLMs to a variety of forecasting tasks and inspect their internal states using sparse autoencoders to understand whether they appear to rely on time-specific pieces of knowledge versus generalizable patterns. Our analyses identify features associated with both time-aware reasoning and look-ahead-biased reasoning. We then apply the LLMs to an entirely different domain and intervene on these features. We find that amplifying time-awareness features substantially reduces look-ahead bias on forecasting prompts while preserving general reasoning performance. In contrast, steering the candidate look-ahead-bias features does not produce an effect. These results suggest that interpretable temporal features can be used to causally shift LLMs toward more historically grounded reasoning.
