Search papers, labs, and topics across Lattice.
This paper introduces SL-FAC, a split learning framework that reduces communication overhead by compressing smashed data. SL-FAC uses Adaptive Frequency Decomposition (AFD) to transform smashed data into the frequency domain, followed by Frequency-based Quantization Compression (FQC) which applies varying quantization bit widths based on spectral energy. Experiments show SL-FAC improves training efficiency by strategically preserving crucial information during compression.
Smashed data bottlenecks in split learning? SL-FAC shrinks communication overhead by intelligently quantizing frequency components, boosting training efficiency.
The growing complexity of neural networks hinders the deployment of distributed machine learning on resource-constrained devices. Split learning (SL) offers a promising solution by partitioning the large model and offloading the primary training workload from edge devices to an edge server. However, the increasing number of participating devices and model complexity leads to significant communication overhead from the transmission of smashed data (e.g., activations and gradients), which constitutes a critical bottleneck for SL. To tackle this challenge, we propose SL-FAC, a communication-efficient SL framework comprising two key components: adaptive frequency decomposition (AFD) and frequency-based quantization compression (FQC). AFD first transforms the smashed data into the frequency domain and decomposes it into spectral components with distinct information. FQC then applies customized quantization bit widths to each component based on its spectral energy distribution. This collaborative approach enables SL-FAC to achieve significant communication reduction while strategically preserving the information most crucial for model convergence. Extensive experiments confirm the superior performance of SL-FAC for improving the training efficiency.
Tsinghua AI