Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates curriculum learning strategies for improving LLM performance on complex Text-to-SQL tasks, specifically addressing challenges with deeply nested statements and noisy database schemas. They find that naive curriculum learning fails due to catastrophic forgetting, and propose a Modular Adapter Composition (MAC) strategy involving sequential training of tier-specific adapters on incrementally complex data. MAC achieves measurable performance gains on Spider and BIRD benchmarks, demonstrating the effectiveness of structured, modular learning for complex code generation.
Forget monolithic fine-tuning: "Lego-Code" shows that modular curriculum learning, by sequentially training adapters on increasingly complex data, significantly boosts LLM performance on complex Text-to-SQL tasks.
Recently, code-oriented large language models (LLMs) have demonstrated strong capabilities in translating natural language into executable code. Text-to-SQL is a significant application of this ability, enabling non-technical users to interact with relational databases using natural language. However, state-of-the-art models continue to struggle with highly complex logic, particularly deeply nested statements involving multiple joins and conditions, as well as with real-world database schemas that are noisy or poorly structured. In this paper, we investigate whether curriculum learning can improve the performance of code-based LLMs on Text-to-SQL tasks. Employing benchmarks including Spider and BIRD, we fine-tune models under different curriculum strategies. Our experiments show that naive curriculum, simply ordering training samples by complexity in a single epoch, fails to surpass standard fine-tuning due to catastrophic forgetting. To overcome this, we propose a Modular Adapter Composition (MAC) strategy. By sequentially training tier-specific adapters on incremental complexity levels (Easy to Extra-Hard), we create a scaffolded learning environment that improves performance on complex queries. Our approach not only produces measurable performance gains on the Spider and BIRD benchmarks but also provides a flexible,"Lego-like"architecture, allowing models to be composed and deployed based on specific schema difficulty requirements. These findings demonstrate that structured, modular learning is a superior alternative to monolithic fine-tuning for mastering the syntax and logic of complex code generation.