Search papers, labs, and topics across Lattice.
The paper introduces a thermodynamic training framework for State Space Models (SSMs) and Transformers, penalizing computational waste alongside cross-entropy loss. Thermodynamically-trained SSMs exhibit "architectural proprioception," characterized by a strong anticipatory coupling between recurrent state entropy and halt confidence, where the halt signal precedes state entropy collapse by two tokens. This "Universal Stopping Signature" (USS) is architecture-dependent, as it is observed in SSMs but not in identically trained Transformers, and it facilitates meta-cognitive transfer learning.
SSMs, unlike Transformers, can be trained to anticipate halting based on internal state entropy, suggesting an inherent architectural advantage for computational self-awareness.
We introduce the Probability Navigation Architecture (PNA) framework, which treats neural computation as navigation through a probability manifold governed by thermodynamic principles. We train State Space Models (SSMs) and Transformers with a novel thermodynamic loss function that penalizes computational waste alongside standard cross-entropy. Across 19 experimental phases, we discover that thermodynamically-trained SSMs develop architectural proprioception: a strong anticipatory coupling between recurrent state entropy and halt confidence (r = -0.836, p < 0.001) in which the halt signal leads state entropy collapse by exactly two tokens (tau = -2.0). This Universal Stopping Signature (USS) reproduces to four decimal places across random seeds and generalizes to a structurally distinct sorting task. Critically, Transformers trained identically show no such coupling (r = -0.07), demonstrating that the phenomenon is architecture-dependent. Cross-task transfer experiments confirm that SSM halt detection reflects genuine meta-cognition (zero-shot transfer F1: SSMs 64.2% vs. Transformers 69.3%; post-adaptation: SSMs 94.5% vs. Transformers 86.4%), while Transformer halt detection relies on syntactic pattern matching. A 2D hyperparameter sweep over energy penalty (alpha) and halt supervision (beta) reveals that the anticipatory coupling is continuously controllable through training, with thermodynamic pressure serving as the primary induction mechanism and explicit halt supervision as an amplifier. Our results establish that SSMs are thermodynamically native architectures whose fixed-size recurrent states naturally support the Markovian compression that enables computational self-awareness, with implications for cost-aware inference, dynamic token budgets, and confidence-based routing in production systems.
