Search papers, labs, and topics across Lattice.
This paper investigates the cryptographic capabilities of transformer networks by mapping specific cryptographic constructions, such as Keccak functions and Merkle Trees, to transformer architectures. By generating threshold circuits for these constructions, the authors establish verified scaling laws for circuit width and depth, leading to two distinct mappings for implementation. The findings yield upper bounds on the computational capacity of transformers, offering a structured approach to evaluate the capabilities of transformer-based AI systems in the context of security and information exchange.
Transformers can be rigorously evaluated for their cryptographic capabilities, revealing upper bounds on their computational power that could redefine security in AI systems.
In recent work it has been shown that colluding AI agents can use steganographic methods to exchange malicious information. Whether a transformer can implement steganographic methods depends on what cryptographic functions it can implement, since a transformer that can implement a cryptographic function within its layers has source-free randomness access. Despite existing circuit-complexity results, no prior work maps specific cryptographic constructions to transformer architectures. As Merrill et al. have shown that saturated transformers can be seen as threshold circuits, we first generate threshold circuits for three different cryptographic constructions (Keccak functions, Merkle--Damgard constructions and Merkle Trees) and then map these circuits to different transformer architectures. We derive verified scaling laws for the width and depth of the circuits which implement each cryptographic construction and propose two different mappings: no-attention mapping, tokens-as-gates mapping. Beyond its security implications, this work contributes to by establishing a methodology for deriving structural guarantees on transformer computational capacity. Specifically, we derive constructive upper bounds on what a transformer of a given depth and width could plausibly compute, providing a principled foundation for capability evaluations of transformer-based AI systems.
