Search papers, labs, and topics across Lattice.
This paper introduces L-shaped split federated fine-tuning (SFT) for large language models, which addresses the communication inefficiencies of conventional federated learning by allowing server-side computation of training loss without requiring continuous client involvement. By leveraging weight tying in LLMs, the method enables direct supervision of hidden activations on the server using target embeddings, thus streamlining the training process. Experimental results show that L-shaped SFT and its one-shot variant significantly reduce communication costs and client online time compared to existing SFT approaches, making it more suitable for edge devices with limited resources.
By eliminating bidirectional communication, L-shaped SFT allows edge devices to participate in fine-tuning without the need for constant connectivity, drastically cutting down on communication overhead.
Large language model (LLM) fine-tuning is increasingly shifting toward data generated on edge devices, where memory, computation, bandwidth, and connectivity constraints make conventional federated learning difficult to sustain. Split federated fine-tuning (SFT) improves client-side efficiency by offloading most model parameters and computation to the server but requires step-by-step bidirectional communication loop across the split interface and forces continuous client involvement throughout training. In this paper, we present L-shaped SFT, a split fine-tuning framework that removes this bidirectional bottleneck. Our key insight is that weight tying in modern LLMs enables server-side hidden activations to be directly supervised using target embeddings, allowing the training loss to be computed on the server without returning server outputs to the client. To further eliminate the need for continuous client participation, based on L-shaped SFT, we introduce one-shot SFT, in which clients upload activations once and then go offline while the server continues optimization over cached representations. We implement our design in a real system testbed with heterogeneous edge clients, including commercial smartphones and NVIDIA developer boards. Experiments demonstrate that our schemes significantly reduce communication costs and client online time compared with existing SFT baselines.
