Search papers, labs, and topics across Lattice.
This paper introduces GAIA, a novel framework for online data selection in instruction tuning of Large Language Models (LLMs) that leverages Gaussian Process regression to model utility across the semantic space. By moving beyond traditional batch-constrained methods, GAIA dynamically prioritizes high-utility samples through an adaptive strategy fusion mechanism, ensuring robust performance even under non-stationary conditions. Empirical results show that GAIA significantly outperforms existing state-of-the-art methods, highlighting its potential for enhancing data quality in LLM training.
GAIA redefines online data selection for LLM instruction tuning, achieving superior performance by dynamically prioritizing high-utility samples across the entire semantic space.
With Large Language Model (LLM) pre-training and fine-tuning shifting its focus from data volume to data quality, quality data selection has emerged as a critical research topic. Existing online data selection methods for LLM training are typically "batch-constrained", limiting optimization to local utility within random batches. To overcome this, we propose GAIA (Global Adaptive Instruction tuning via GAussian processes), a framework that formulates data valuation as a global estimation process. GAIA employs Gaussian Process regression to model continuous utility manifolds across the semantic space, utilizing an adaptive strategy fusion mechanism to dynamically prioritize high-utility samples. By casting the strategy-posterior update as an instance of the classical fixed-share Hedge framework for tracking the best expert, we inherit a dynamic-regret guarantee that characterizes GAIA's robustness under non-stationary quality scores during training. Empirical evaluations on three datasets demonstrate that GAIA significantly outperforms state-of-the-art baselines like \greats, establishing our method as a scalable and robust solution for efficient instruction tuning.
