Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
We analyze the trade-off between noise reduction and bias in federated learning for least-squares models using Generalized Total Variation Minimization (GTVMin) in a large-dimensional regime. Employing random matrix theory, we derive exact analytic expressions for the asymptotic performance on both i.i.d. and non-i.i.d. data distributions, revealing how regularization balances local adaptation with global collaboration. The analysis shows that Lipschitz observations of the GTVMin solution concentrate universally across network structures, providing a theoretical foundation for understanding federated learning performance.
Federated learning's bias-variance trade-off is precisely quantified in high dimensions, revealing how GTVMin regularization optimally balances local adaptation and global collaboration.
Federated learning over decentralized networks inherently trades off the suppression of local finite-sample noise through collaboration against the bias that collaboration introduces between heterogeneous nodes. We study this tension by performing a large-dimensional analysis of Generalized Total Variation Minimization with least-squares models. Using tools from random matrix theory, we show that Lipschitz observations of the global GTVMin solution concentrate in large dimensions, universally for any underlying network structure. We provide exact analytic expressions for the asymptotically precise performance under both i.i.d. and non-i.i.d. Gaussian mixture distributions on a fixed network. For the latter, these expressions quantify how regularization balances local adaptation against collaboration. Our theoretical results are confirmed by experiments on synthetic and real-world data.