Search papers, labs, and topics across Lattice.
This paper introduces GAPSL, a gradient-aligned parallel split learning framework designed to mitigate training divergence in heterogeneous federated learning settings. GAPSL uses Leader Gradient Identification (LGI) to select directionally consistent client gradients and Gradient Direction Alignment (GDA) to regularize client gradients towards a global convergence trend. Experiments on a computing testbed show GAPSL achieves superior training accuracy and reduced latency compared to existing parallel split learning methods.
Gradient misalignment across devices in parallel split learning can be tamed with a novel gradient alignment strategy, leading to faster convergence and higher accuracy in heterogeneous federated learning.
The increasing complexity of neural networks poses significant challenges for democratizing FL on resource?constrained client devices. Parallel split learning (PSL) has emerged as a promising solution by offloading substantial computing workload to a server via model partitioning, shrinking client-side computing load, and eliminating the client-side model aggregation for reduced communication and deployment costs. Since PSL is aggregation-free, it suffers from severe training divergence stemming from gradient directional inconsistency across clients. To address this challenge, we propose GAPSL, a gradient-aligned PSL framework that comprises two key components: leader gradient identification (LGI) and gradient direction alignment (GDA). LGI dynamically selects a set of directionally consistent client gradients to construct a leader gradient that captures the global convergence trend. GDA employs a direction-aware regularization to align each client's gradient with the leader gradient, thereby mitigating inter-device gradient directional inconsistency and enhancing model convergence. We evaluate GAPSL on a prototype computing testbed. Extensive experiments demonstrate that GAPSL consistently outperforms state-of-the-art benchmarks in training accuracy and latency.
