Search papers, labs, and topics across Lattice.
This study investigates the trade-offs in task accuracy when applying parameter-efficient fine-tuning (PEFT) and low-bit quantization on a 60M-parameter T5-small model for text-to-SQL tasks. By systematically varying LoRA rank, adapted modules, and numerical precision, the authors reveal that a LoRA rank of 16 achieves 59.6% accuracy with less than 1% of parameters trained and 31% reduced peak GPU memory usage. Additionally, QLoRA with INT8 and NF4 quantization maintains competitive accuracy while drastically lowering memory requirements, highlighting the efficiency of these methods in resource-constrained environments.
Achieving 59.6% accuracy with LoRA rank 16 while training less than 1% of parameters showcases the potential for extreme efficiency in fine-tuning language models.
Parameter-efficient fine-tuning (PEFT) and low-bit quantization are now standard tools for adapting language models under tight compute budgets, yet their interaction is most often studied on billion-parameter models where the design space is expensive to explore. We ask a complementary question: on a specific, fully reproducible 60M-parameter encoder-decoder model (T5-small) and a single-table text-to-SQL benchmark (WikiSQL), how much task accuracy does each efficiency knob actually cost? We run a controlled, single-variable study over (i) LoRA rank r in {2, 4, 8, 16, 32}, (ii) the set of adapted modules, and (iii) numerical precision. We report task accuracy alongside system-level metrics including trainable parameters, peak training memory, inference latency, and throughput, and frame adaptation as a constrained trade-off rather than an accuracy-only objective. Our results show that LoRA with r=16 recovers within 11.6 percentage points of full fine-tuning accuracy (59.6% vs. 71.2% exact-match) while training fewer than 1% of parameters and consuming 31% less peak GPU memory. Within this setting, rank beyond r=16 yields no measurable accuracy gain. QLoRA with INT8 and NF4 quantization achieves comparable accuracy (52.8% and 53.2%) at dramatically lower memory cost (0.60 GB each), demonstrating a compelling trade-off for memory-constrained deployments. All code, configurations, and logs are released for full reproducibility.
