Search papers, labs, and topics across Lattice.
This paper introduces FedCC, an innovative algorithm designed to address label distribution skews in distillation-based federated learning (FL) by allowing clients to tag ambiguous samples as 'unknown'. This approach mitigates the risk of error propagation from misclassifications and enhances the server's ability to calibrate predictions through the use of calibrated pseudo-labels on a public dataset. Experimental results show that FedCC significantly outperforms existing methods, achieving 67.3% accuracy in extreme scenarios of label skew, while traditional methods falter to near-random performance.
FedCC enables clients to handle ambiguity in data, leading to a remarkable 67.3% accuracy even when facing severe label distribution skews.
Federated Learning (FL) enables distributed clients to collaboratively train models without sharing raw data, making it promising for leveraging massive devices in communication networks. In distillation-based FL, each client applies its local model on an unlabeled public dataset, and shares only prediction results with the server. While heterogeneous local data introduces label distribution skew, thus biasing client models toward majority classes and leading to potentially inaccurate predictions. The lack of ground-truth labels in the public dataset hampers the server's ability to calibrate predictions, which ultimately degrades overall performance. To address this, we propose FedCC, a simple and effective algorithm for mitigating client misclassification. Instead of being forced to classify and risking error propagation, clients are allowed to tag ambiguous samples as 'unknown'. This additional class, together with calibrated pseudo-labels on the public data, balances confidence in majority classes against uncertainty in under-represented ones. Extensive experiments demonstrate that FedCC significantly outperforms existing methods, especially under severe label skew. In the extreme scenario where each client holds samples from only one of ten classes, FedCC achieves 67.3% accuracy, while baselines collapse to near-random results.
