Search papers, labs, and topics across Lattice.
This paper introduces AEGIS, a novel defense mechanism designed to mitigate gradient inversion attacks in federated learning by isolating three channels through which private token information can leak. By employing a combination of freezing attention projection parameters, injecting calibrated noise into embedding gradients, and applying noise to MLP expansion gradients, AEGIS effectively eliminates the risk of token recovery while maintaining or enhancing model utility. Evaluations across multiple models and datasets demonstrate that AEGIS reduces token recovery rates to near zero against various attack strategies, providing formal guarantees for its effectiveness.
AEGIS slashes token recovery rates to nearly zero while preserving model performance, tackling all three critical channels of information leakage in federated learning.
Gradient inversion attacks recover private training text from gradients shared in federated learning, posing a serious threat to collaborative model training. Through our analysis of transformer gradient structure, we identify three channels through which private token information leaks: the attention output projection gradient exposes a low-rank subspace that encodes input embeddings (Channel 1), the embedding gradient's row-norm sparsity directly reveals which tokens are present (Channel 2), and the MLP expansion gradient carries a recoverable subspace signal analogous to Channel 1 (Channel 3). State-of-the-art attacks exploit these channels analytically to achieve near-exact token recovery in seconds. Existing defences address at most one channel and either degrade model utility or leave the remaining structural signals intact. We introduce AEGIS (Attention-Embedding Gradient Isolation Shield), a lightweight defence that closes all three analytical channels with three backward-path operations requiring no architectural changes: freezing attention projection parameters eliminates Channel 1 by construction, calibrated noise injection into the embedding gradient destroys Channel 2's token-presence signal, and analogous per-block noise injection into the MLP expansion gradient masks Channel 3. The same masked gradient drives both the local optimiser step and the server export, so no clean signal is retained on either side. Evaluated across 11 models and six datasets, AEGIS reduces token recovery rates to near zero against a range of gradient inversion attacks, both analytical and optimisation-based, while preserving or improving model utility. We provide formal guarantees for Channels 1 and 2 and validate the full defence empirically against adaptive adversaries with complete knowledge of the mechanism.
