Search papers, labs, and topics across Lattice.
This paper introduces a novel method for ensuring the integrity of peer-to-peer distributed LLM inference by measuring variations in activations across nodes, effectively identifying malicious tampering. By utilizing secret canary inputs mixed with regular traffic, the approach distinguishes benign nodes from tampered ones based on their output variations, achieving an impressive AUROC of 1.0 across 408 configurations. This advancement is crucial as it allows for robust integrity checks without the computational overhead associated with traditional methods like recomputing on trusted hardware.
Malicious nodes can be detected with 100% accuracy by measuring activation variations in a peer-to-peer LLM inference network, revolutionizing trust in distributed AI systems.
Peer-to-peer distributed inference executes a Large Language Model (LLM) on pooled consumer hardware by spreading its layers across many nodes. Every request passes through nodes that are owned and controlled by multiple independent parties. However, in this setting, any party can tamper with the output of its layers to corrupt the end result. Recomputing the forward pass on trusted hardware can catch this, but it introduces additional computational cost. The scientific literature includes several prior integrity-checking approaches, such as known-answer traps for image classifiers and cryptographic commitments. However, these solutions test only the exact correctness and do not account for the ordinary variation that may arise between benign nodes. In this paper, we propose a method that checks the output integrity by measuring the variation in the activations that each node passes to the next. A peer who wants to use the network selects a small set of secret canary inputs whose correct activations are known in advance and mixes them into regular traffic. Because the peers cannot tell a canary from a real query, any tampering node corrupts them as well. The deviation from the known reference then reveals malicious activity: benign nodes exhibit only minor variation from hardware-induced noise, whereas tampered nodes deviate far more. We treat the identification of malicious nodes as a probabilistic test that separates two drift distributions, without relying on a fixed threshold. We study 408 configurations with metrics and success criteria fixed before any experiment ran; the detector reaches AUROC 1.0, correctly ranking the malicious shard above every benign shard on every canary in every configuration.
