Search papers, labs, and topics across Lattice.
This paper examines the feasibility of evading compute governance regulations by leveraging distributed training to conduct frontier AI training on geographically dispersed hardware. It argues that advances in distributed training algorithms could allow developers to bypass regulations tied to large, centralized computing clusters. The paper then proposes countermeasures like whistleblowing, chip tracking, and forensic accounting to detect and prevent illicit distributed training operations.
Compute governance could be undermined by advances in distributed training, enabling frontier AI development outside the reach of centralized oversight.
Compute governance proposals often rely on the assumption that frontier AI training requires large, detectable computing clusters. However, recent advances in distributed training algorithms could allow developers to conduct frontier-scale training on distributed agglomerations of hardware, rather than needing large datacenter facilities. Developers who prefer not to be constrained by regulations may structure their hardware in a manner that evades the registration and monitoring requirements associated with compute governance. Therefore, regulations must be designed to detect and prevent illicit distributed training operations. This paper evaluates the feasibility of such evasion and outlines recommended countermeasures, including whistleblowing, chip tracking, forensic accounting, and memory and compute thresholds for clusters.
Stanford HAI