Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper analyzes a mixed-regime process where a sequence model must predict text generated by either a deterministic textual regime or a random regime governed by a hidden state. It demonstrates that even perfect sequence predictors can become overconfident when the observed prefix aligns with the incorrect latent regime, creating a "sufficiency gap" due to marginalization. The authors then formalize how external grounding signals can correct this, deriving a contextual dominance threshold that dictates when the signal reverses the posterior odds induced by misleading text, highlighting the need for decoupled observers in autonomous sequence models.
Sequence models can be confidently wrong even when they perfectly capture the text-only distribution, revealing a fundamental "sufficiency gap" that necessitates external verification.
We construct a binary mixed-regime process with one deterministic textual regime and one random regime governed by an unobserved latent state. Even an ideal infinite-capacity sequence predictor that exactly recovers the text-only marginal law can become overconfident when the observed prefix is compatible with the wrong latent regime. The resulting entropy difference is not an ordinary optimization error; it is a sufficiency gap caused by marginalization over an unobserved state. We then formalize retrieval, tool use, and external grounding through an auxiliary binary signal with fidelity $γ\in [1/2,1]$. The resulting Bayesian update yields a contextual dominance threshold: a corrective signal reverses the posterior odds induced by the textual history exactly when its fidelity exceeds the text-only posterior weight assigned to the misleading regime. This threshold reduces, but does not generally eliminate, the sufficiency gap; complete closure requires perfect revelation of the relevant latent state or an equivalent verification mechanism. The analysis clarifies why temperature scaling cannot restore missing context, why grounding mechanisms must be both informative and learnably usable by the model, and why autonomous sequence models require structurally decoupled observers or verifiers in high-stakes domains.