Search papers, labs, and topics across Lattice.
This paper introduces a novel approach to Clustered Federated Learning by utilizing Random Network Distillation to identify client similarities based on novelty signals derived from local data. By decoupling the clustering process from the main training loop, the method significantly reduces computational and communication overhead while enabling the formation of client groups without prior knowledge of their structure. The key finding is that this lightweight clustering mechanism allows for efficient and autonomous collaboration in large-scale distributed systems facing non-iid data challenges.
Clustering clients based on local novelty signals can revolutionize federated learning by enabling efficient and autonomous collaboration without the need for extensive computational resources.
Federated Learning often suffers under non-independently and identically distributed data, where a single global model may fail to represent the diversity of client distributions. Clustered Federated Learning mitigates this issue by training specialized models for groups of similar clients, but existing approaches often couple cluster assignment with the main training loop, increasing computational and communication costs. We propose a lightweight clustering approach based on Random Network Distillation. Each client trains a compact Random Network Distillation predictor on its local data and uses its prediction error as a novelty signal to estimate similarity with other clients. This enables the discovery of meaningful client groups before federated training, without sharing raw data or repeatedly evaluating the main model. Crucially, the resulting federations emerge from local novelty estimates at runtime, making the method suitable for autonomous large-scale distributed systems where neither the number of clusters nor the collaboration structure can be specified a priori. Overall, by decoupling clustering from learning, the method provides a task-agnostic and efficient mechanism for autonomous collaboration under non-independently and identically distributed data.
