Search papers, labs, and topics across Lattice.
This paper investigates the fine-tuning behavior of various small language models (SLMs) on resource-constrained embedded GPU platforms, focusing on the energy demands of repeated forward-backward optimization. By characterizing the performance of BERT and Pythia variants on GLUE benchmarks, the authors identify optimal energy-efficient settings for Dynamic Voltage Frequency Scaling (DVFS). Their approach achieves an average energy savings of 13.11%, with a maximum of 26.73%, compared to traditional power management modes on the NVIDIA Jetson AGX Orin.
Fine-tuning small language models can be 26.73% more energy-efficient with optimal DVFS settings on embedded GPUs.
Dynamic Voltage Frequency Scaling (DVFS) on resource-constrained embedded GPU platforms is essential for energy-efficient small language model (SLM) fine-tuning, as privacy- and personalization-driven adaptation increasingly requires local execution and involves repeated forward-backward optimization over many mini-batches, making it substantially more time- and energy-intensive than single-pass inference. To this end, 1) we first characterize the fine-tuning behavior of representative encoder-only SLMs of BERT variants, and autoregressive decoder-only SLMs of Pythia variants on GLUE benchmarks. In addition to the characterizations, 2) we propose a simple yet effective ML-based model selection that selects energy-optimal GPU DVFS settings on resource-constrained embedded platforms. Our results on NVIDIA Jetson AGX Orin demonstrate average 13.11% energy savings (up to 26.73%) over MAXN Mode 0, which has no explicit power cap.
