Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper categorizes gradient and model compression techniques in federated learning (FL) based on the type of correlation they exploit: structural, temporal, and spatial. They propose quantitative metrics to measure the magnitude of these correlations and reinterpret existing compression methods within this framework. Through experiments, they show that the degree of each correlation type varies significantly with task, model, and algorithm, motivating adaptive compression schemes that switch modes based on measured correlation strength.
Blindly applying compression techniques in federated learning can be suboptimal, as the strength of underlying data correlations varies significantly across tasks, models, and algorithms.
The communication bottleneck in federated learning (FL) has spurred extensive research into techniques to reduce the volume of data exchanged between client devices and the central parameter server. In this paper, we systematically classify gradient and model compression schemes into three categories based on the type of correlations they exploit: structural, temporal, and spatial. We examine the sources of such correlations, propose quantitative metrics for measuring their magnitude, and reinterpret existing compression methods through this unified correlation-based framework. Our experimental studies demonstrate that the degrees of structural, temporal, and spatial correlations vary significantly depending on task complexity, model architecture, and algorithmic configurations. These findings suggest that algorithm designers should carefully evaluate correlation assumptions under specific deployment scenarios rather than assuming that they are always present. Motivated by these findings, we propose two adaptive compression designs that actively switch between different compression modes based on the measured correlation strength, and we evaluate their performance gains relative to conventional non-adaptive approaches. In summary, our unified taxonomy provides a clean and principled foundation for developing more effective and application-specific compression techniques for FL systems.