Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces predictive sufficiency to quantify the information gain from external context in model-based RL, decomposing it into history-recoverable and context-dependent components. The authors demonstrate that the necessity of external context for prediction is not solely determined by the MDP class but also by the agent's induced visitation distribution and prior beliefs. When predictive headroom persists, learned states only partially expose it, and external context remains beneficial.
External context is often unnecessary for model-based RL agents that implicitly learn to recover latent environment dynamics from their own experience.
Methods for generalization in model-based reinforcement learning typically assume that an agent cannot recover the latent context governing the environment dynamics from its own experience, and therefore supplies it externally. We formalize and test this assumption with \emph{predictive sufficiency}, which quantifies what access to the context adds to next-step prediction under the visitation distribution an agent induces, and separates that quantity into a history-recoverable part, a residual requiring the true context, and the deficit added by a finite model. We classify context-aware algorithms by the predictive risk their conditioning set can target and demonstrate across environments of increasing identification difficulty that the headroom does not follow the MDP class. The same task under different priors leaves predictive headroom in one setting and nothing distinguishable from zero in another, where the agent's behavior implicitly identifies the context and any benefit of such a mechanism cannot be attributed to missing information. Where headroom persists, the learned state exposes it only partially, and adding the true context still lowers the risk. Our contribution is a practical criterion for matching contextual mechanisms to the information available to them, estimated from the ordinary trained agent without a reference policy.