Search papers, labs, and topics across Lattice.
This paper addresses the governance challenges posed by self-adapting generative AI systems that continuously update their weights during deployment, which undermines traditional model risk management (MRM) practices. It develops a telemetry architecture that combines discrete and continuous time logging methods, including a Minimal Sufficient Statistic for audits and a tamper-evident Merkle chain for weight sequences, while also proposing event-driven logging based on KL divergence. Furthermore, it explores the Model Hiding Problem, categorizing six adversarial strategies that could be employed to conceal updates and offering formal countermeasures for each, thereby emphasizing the need for periodic invasive audits alongside continuous telemetry.
Self-adapting AI models can evade traditional validation checks, but a new dual-regime architecture reveals how to audit them effectively against adversarial concealment tactics.
Model risk management (MRM) guidance assumes a static model lifecycle, in which models are developed, independently validated, and implemented without further autonomous modification. Continually self-adapting generative AI systems --- models that update their own weights during production deployment --- fundamentally violate this assumption and render point-in-time validation inadequate. This paper addresses the resulting governance problem in two parts. Part I develops a rigorous telemetry architecture for such models, operating simultaneously in discrete and continuous time. We establish a Minimal Sufficient Statistic for audit purposes, construct a tamper-evident Merkle chain for discrete weight sequences, derive the appropriate continuous-time generalization via the Ito formula, and propose event-driven logging via KL divergence stopping times that is both computationally tractable and meaningful for validation. Part II asks what happens when the model provider is adversarial. A firm deploying such a model has strong incentives to conceal learning updates that would trigger mandatory validation review. We formalize this as the Model Hiding Problem and provide a systematic taxonomy of six distinct attack strategies against the Part I architecture, spanning discrete and continuous time, with a formal countermeasure for each. Together the two parts establish a dual-regime architecture in which continuous telemetry is necessary but not sufficient, narrowing---but never replacing---periodic invasive audit. The framework is model-architecture-agnostic and is designed to satisfy the three pillars of traditional MRM.
