Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper addresses the challenge of gradient leakage in decentralized federated learning by introducing a communication-efficient secure aggregation scheme that leverages top-K gradient sparsification. By offloading dimension-dependent overhead to an offline phase and employing random masks and permutations, the proposed method effectively mitigates the unpredictability of user-selected top-K entries while maintaining model accuracy. Experimental results show that the scheme achieves comparable accuracy to full-gradient aggregation with only 1% gradient sparsification, significantly reducing communication costs.
Achieving competitive model accuracy with only 1% gradient transmission could revolutionize communication efficiency in decentralized federated learning.
Secure aggregation is a vital component for mitigating gradient leakage in federated learning, but its communication cost conventionally scales with the gradient dimension. This becomes prohibitive for large models and even more pronounced in decentralized federated learning with limited bandwidth and unreliable nodes. Top-K gradient sparsification is an effective approach to reduce communication by transmitting only a few entries of the full gradient, while maintaining competitive model accuracy. Nevertheless, the top-K entries selected by each user are unpredictable and vary across users, which poses a challenge for efficient sparse secure aggregation. This paper studies information-theoretic secure aggregation with top-K sparsification in decentralized federated learning under user dropouts and user collusion. We propose a communication-efficient sparse secure aggregation scheme that offloads dimension-dependent overhead to an offline phase and protects private gradients using random masks and permutations. Experimental results demonstrate that our scheme preserves accuracy comparable to full-gradient aggregation even with only 1% gradient sparsification, while substantially reducing the communication cost.