Search papers, labs, and topics across Lattice.
This paper introduces Collate, a collaborative training framework designed for Federated Learning (FL) that addresses the challenges of heterogeneous devices in latency-critical edge systems. By employing a dynamic zeroizing-recovering method, it optimizes local model architectures to meet specific latency constraints while enhancing accuracy. Experimental results demonstrate that Collate achieves an average accuracy improvement of 1.96% for extended models and 3.09% for shrunk models, all with minimal additional training overhead.
Heterogeneous models can be collaboratively trained to improve accuracy under strict latency constraints, achieving significant performance gains without extra training costs.
Federated Learning (FL) empowers multiple clients to collaboratively learn a model, enlarging the training data of each client for high accuracy while protecting data privacy. However, when deploying FL in real-time edge systems, the heterogeneity of devices among systems has a severe impact on the performance of the inferred model. Existing optimizations on FL focus on improving the training efficiency but fail to speed up inference, especially when there is a latency constraint. In this work, we propose Collate, a novel training framework that collaboratively learns heterogeneous models to meet the latency constraints of multiple edge systems simultaneously. We design a dynamic zeroizing-recovering method to adjust each local model architecture for high accuracy under its latency constraint. A proto-corrected federated aggregation scheme is also introduced to aggregate all heterogeneous local models, satisfying the latency constraint of different systems with only one training process and maintaining high accuracy. Extensive experiments indicate that, compared to state-of-the-art methods and under a latency constraint, our extended models can improve the accuracy by 1.96% on average, and our shrunk models can also obtain a 3.09% accuracy improvement on average, with almost no extra training overhead. The related codes and data will be available at https://github.com/ntuliuteam/Collate.
