Search papers, labs, and topics across Lattice.
SubFLOT addresses system and statistical heterogeneity in federated learning by introducing a server-side personalized pruning framework. It uses an Optimal Transport-enhanced Pruning (OTP) module to generate customized submodels based on historical client models, minimizing the Wasserstein distance between submodels and client data distributions. Additionally, a Scaling-based Adaptive Regularization (SAR) module penalizes submodel deviation from the global model, scaled by the client's pruning rate, to counteract parametric divergence. Experiments show SubFLOT outperforms state-of-the-art methods, enabling efficient and personalized models on edge devices.
SubFLOT tackles federated learning's heterogeneity problem by cleverly using optimal transport to create personalized submodels on the server, sidestepping the computational burden of client-side pruning.
Federated Learning (FL) enables collaborative model training while preserving data privacy, but its practical deployment is hampered by system and statistical heterogeneity. While federated network pruning offers a path to mitigate these issues, existing methods face a critical dilemma: server-side pruning lacks personalization, whereas client-side pruning is computationally prohibitive for resource-constrained devices. Furthermore, the pruning process itself induces significant parametric divergence among heterogeneous submodels, destabilizing training and hindering global convergence. To address these challenges, we propose SubFLOT, a novel framework for server-side personalized federated pruning. SubFLOT introduces an Optimal Transport-enhanced Pruning (OTP) module that treats historical client models as proxies for local data distributions, formulating the pruning task as a Wasserstein distance minimization problem to generate customized submodels without accessing raw data. Concurrently, to counteract parametric divergence, our Scaling-based Adaptive Regularization (SAR) module adaptively penalizes a submodel's deviation from the global model, with the penalty's strength scaled by the client's pruning rate. Comprehensive experiments demonstrate that SubFLOT consistently and substantially outperforms state-of-the-art methods, underscoring its potential for deploying efficient and personalized models on resource-constrained edge devices.
Tsinghua AI