Search papers, labs, and topics across Lattice.
This paper introduces PaME, a decentralized federated learning (DFL) algorithm that reduces communication costs and enhances privacy by exchanging only randomly selected sparse coordinates between neighboring nodes. PaME achieves linear convergence under mild assumptions of locally Lipschitz continuous gradients and a doubly stochastic communication matrix, relaxing restrictive conditions common in existing DFL methods. Empirical results demonstrate PaME's superior performance compared to other decentralized learning algorithms, highlighting its effectiveness in handling data heterogeneity.
DFL gets a communication and privacy boost: PaME exchanges only sparse coordinates, achieving linear convergence under milder assumptions than existing methods.
Decentralized federated learning (DFL) has emerged as a transformative server-free paradigm that enables collaborative learning over large-scale heterogeneous networks. However, it continues to face fundamental challenges, including data heterogeneity, restrictive assumptions for theoretical analysis, and degraded convergence when standard communication- or privacyenhancing techniques are applied. To overcome these drawbacks, this paper develops a novel algorithm, PaME (DFL by Partial Message Exchange). The central principle is to allow only randomly selected sparse coordinates to be exchanged between two neighbor nodes. Consequently, PaME achieves substantial reductions in communication costs while still preserving a high level of privacy, without sacrificing accuracy. Moreover, grounded in rigorous analysis, the algorithm is shown to converge at a linear rate under the gradient to be locally Lipschitz continuous and the communication matrix to be doubly stochastic. These two mild assumptions not only dispense with many restrictive conditions commonly imposed by existing DFL methods but also enables PaME to effectively address data heterogeneity. Furthermore, comprehensive numerical experiments demonstrate its superior performance compared with several representative decentralized learning algorithms.
