Search papers, labs, and topics across Lattice.
This paper introduces LIGHTYEAR, a federated learning framework that enhances update selection by utilizing an NTK-based agreement score to evaluate predictive behavior in function space rather than parameter space. By allowing clients to directly exchange updates and assess their impact on local validation data, LIGHTYEAR effectively mitigates the negative effects of heterogeneous data and malfunctioning clients. The results demonstrate that LIGHTYEAR consistently outperforms traditional centralized federated learning approaches and existing peer-to-peer methods across multiple datasets, highlighting its robustness in diverse training environments.
Function-space analysis reveals that traditional parameter comparisons can lead to suboptimal updates in federated learning, with LIGHTYEAR showing significant performance gains.
Federated learning (FL) enables collaborative model training across distributed clients while keeping data local. A central challenge is determining which client updates are beneficial for aggregation with respect to each client's target domain. Existing methods typically address this problem in parameter space by comparing model parameters or gradients. However, parameter-space similarity can be a poor proxy for predictive behavior, especially under heterogeneous, non-IID data. Consequently, updates that are misaligned with a client's target domain, including those caused by heterogeneous data or malfunctioning clients, may degrade local model performance. We propose Local Inference Guided Aggregation for Heterogeneous Training Environments to Yield Enhancement Through Agreement and Regularization (LIGHTYEAR), a federated learning framework that performs update selection in function space. LIGHTYEAR uses an NTK-based agreement score to characterize predictive behavior and determine a personalized aggregation set for each client. By relating model parameters to local predictive responses, the Neural Tangent Kernel (NTK) provides a more expressive criterion for update selection than parameter-space similarity alone. Because function-space information is not available before aggregation in conventional centralized FL, LIGHTYEAR uses a peer-to-peer (P2P) topology in which clients exchange updates directly and evaluate incoming models on private validation data. Each client selects only updates that are beneficial for its own target domain and aggregates them using a regularized rule that improves stability under heterogeneity. Across five datasets and nine baseline methods, LIGHTYEAR consistently outperforms centralized FL baselines and existing P2P approaches.
