Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a novel spectral federated learning method that integrates communication compression and local steps, overcoming limitations of prior spectral optimizer adaptations. The approach establishes rigorous convergence guarantees without requiring bounded heterogeneity, a significant theoretical advancement. Empirical results on image classification benchmarks demonstrate the method's efficiency and robustness.
Spectral optimization finally bridges compressed communication and local steps in federated learning, achieving convergence without bounded heterogeneity.
Federated Learning (FL) enables collaborative training of complex models across large-scale networks of edge devices while keeping data localized. However, data heterogeneity and high communication overhead remain fundamental bottlenecks. Recently, the spectral optimizer Muon has demonstrated remarkable advantages over Adam-like methods in centralized training. While recent efforts have attempted to transition Muon to FL, naive integrations into traditional architectures fail short. Crucially, existing approaches do not support the the communication compression and local steps --- the two foundational techniques of efficient FL. To address this limitation, we introduce a novel spectral FL method that utilizes both local updates and compression. We establish rigorous convergence guarantees under mild assumptions, notably eliminating the restrictive bounded heterogeneity condition. Empirical evaluation on standard image classification benchmarks validates the efficiency and robustness of our approach.