Search papers, labs, and topics across Lattice.
This study investigates the integration of a calibrated, differentiable energy surrogate into the fine-tuning objective of Large Language Models (LLMs) to optimize the balance between task accuracy and carbon emissions during inference. By employing a joint loss mechanism that incorporates a carbon-emission parameter and various proxies for model efficiency, the authors fine-tune three distinct LLM architectures and assess their performance on multiple subjects. The findings reveal that the carbon term can either hinder or enhance model performance depending on the specific task, indicating a nuanced approach to achieving greener LLMs with potential for practical application.
Fine-tuning LLMs with a carbon-aware objective can yield task accuracy improvements while minimizing carbon emissions, but the effectiveness varies by task structure.
The carbon footprint of any deployed Large Language Model (LLM) accumulates during inference, where repeated use of the model substantially exceeds the one-time cost of fine-tuning. Yet most efficiency interventions target either pre-training scale or post-hoc compression. We ask whether folding a calibrated, differentiable energy surrogate into the fine-tuning objective can produce inference behavior that gains task accuracy at zero or near-zero carbon cost, a break-even configuration. We propose a joint loss mechanism with a per-model carbon-emission parameter, a linear surrogate over parameter norm, FLOP proxy, and a memory proxy, fit from on-hardware energy profiling. We fine-tune three architecturally distinct families: Gemma-2 2B, Llama-3.1 8B, and Qwen-2.5 14B, and evaluate inference F1 and CO$_2$ emissions on three MMLU subjects: abstract algebra, philosophy, and formal logic. We discover from several outcomes that the carbon term behaves as either harmful interference or beneficial regularization depending on the task structure. We position calibrated carbon-aware fine-tuning as a lightweight, drop-in regularizer with a non-empty but model and task-dependent break-even region. This is an ongoing work, and we will release our codebase soon.
