Search papers, labs, and topics across Lattice.
This paper develops a comprehensive framework for evaluating AI systems within controlled environments, termed AI sandboxes, which are essential for testing and validating AI in physical and cyber-physical contexts. It introduces a formalized threat model and taxonomy that delineates various sandbox archetypes, alongside a measurement framework that assesses key dimensions such as fidelity and observability. The findings clarify the capabilities and limitations of AI sandboxes, providing crucial insights into their role in ensuring safety and regulatory compliance in AI deployments.
AI sandboxes can be rigorously defined to expose vulnerabilities in assurance mechanisms, revealing what risks they can truly contain.
AI systems are increasingly evaluated in bounded environments that combine isolation, simulation, instrumentation, supervision, and evidence capture. For physical AI, AIoT, and cyber-physical systems, this shift is not a matter of terminology: the system under test may sense, decide, actuate, communicate, and fail through physical processes, networked devices, and human operators. This article develops an assurance-oriented account of AI sandboxes as controlled environments for testing, evaluation, verification, and validation across digital AI, embodied autonomy, and cyber-physical deployments. We formalize the sandbox boundary and a weakest-link rule for composing per-dimension evidence into a bounded deployment claim; separate major sandbox archetypes; define a cyber-physical threat model that includes attacks on the assurance apparatus itself; and introduce a measurement framework spanning fidelity, controllability, observability, containment, reproducibility, and governance artifacts, instantiated on three worked case studies of real sandboxes. The resulting threat model, taxonomy, and measurement framework clarify what a sandbox can validly test, which risks it can contain, and what forms of evidence it can support for safety, security, and regulatory assurance.
