Search papers, labs, and topics across Lattice.
This paper introduces FEAST, a federated shared-space training framework designed to optimize model deployment for resource-heterogeneous clients in federated learning (FL). By jointly training multiple subnetworks tailored to individual client inference budgets and employing budget-tailored sub-supernet routing, FEAST significantly enhances accuracy while reducing parameter transmission traffic. Experimental results demonstrate that FEAST achieves a population-averaged accuracy of 71.06% across multiple datasets, outperforming existing weight-sharing methods and addressing the challenges posed by varying computational capabilities among devices.
FEAST boosts federated learning accuracy by over 2.4 points compared to the best existing model-heterogeneous weight-sharing baseline, all while slashing parameter traffic by 6.8 times.
Federated learning (FL) must serve devices with varying computational capabilities. A fixed model cannot suit all devices, while training one model per deployment limit is costly. Federated supernet training instead learns one elastic model with differently sized subnetworks, then deploys a suitable one to each device. When client inference budgets differ, however, parameters exclusive to high-cost subnetworks are reachable by fewer clients. We propose FEAST, a federated shared-space training framework that counters this imbalance by jointly training multiple subnetworks within each client's limit. Budget-tailored sub-supernet routing sends only the relevant supernet portion, and sparse aggregation merges the returned parameter slices. The trained supernet directly serves the subnetworks used during federation and supports post-hoc extraction of additional subnetworks without federated retraining. We further show that independently assigning clients'training-data volumes and inference budgets can distort accuracy--inference-cost comparisons in heterogeneous FL simulations, and introduce a one-parameter $\gamma$-allocation protocol to control this coupling. In our experimental setup, the SuperFedNAS and DeepFedNAS supernet training procedures remain near chance at 25M and reach at most $17.09\%$ at $596$M inference MACs; FEAST reaches $71.06\%$ at $596$M, $2.4$ points above the strongest model-heterogeneous weight-sharing baseline at its largest tier. Across CIFAR-100, CINIC-10, and TinyImageNet-200, FEAST achieves the highest population-averaged accuracy among the evaluated weight-sharing methods when each client receives its largest affordable subnetwork. Sub-supernet routing reduces aggregate model-parameter traffic by $6.8\times$ relative to full-supernet transmission.
