Track new results across pure and applied mathematics.
Search papers, labs, and topics across Lattice.
This paper introduces Structured Recipe Optimization (SRO), a novel paradigm for automated post-training of LLMs that adapts the search space of training recipes to target tasks. SRO comprises an LLM-based Recipe-Space Designer and a Multi-Domain Recipe Optimizer that leverages structured trial evidence for efficient exploration. Experiments demonstrate SRO significantly outperforms baselines in joint post-training across multiple domains and benchmarks, achieving substantially higher scores with larger models.
LLM agents can now dynamically design their own post-training recipe spaces, leading to a 2.6x performance boost on complex multi-domain tasks.
Automated post-training transforms pre-trained large language models (LLMs) into models capable of handling diverse domains by automatically constructing and optimizing a recipe of coupled decisions over data sources, data processing, cross-domain composition, and training configuration under a limited experimental budget. Existing automation either searches predefined recipe spaces that enable systematic optimization but remain fixed rather than adapting their available choices to the target tasks, or uses open-ended agents that discover task-specific choices but leave them implicit in trial-level edits, limiting their systematic reuse and recombination across trials. To fill this gap, we introduce optimization over a task-adapted recipe space, a novel paradigm that can adapt the recipe space itself to the target tasks while representing its choices explicitly for systematic reuse, recombination, and joint optimization. We instantiate this paradigm with Structured Recipe Optimization (SRO), comprising an LLM-based Recipe-Space Designer that constructs the task-adapted recipe space and a Multi-Domain Recipe Optimizer that uses structured trial evidence to guide LLM candidate construction and multi-objective Bayesian trial selection. Extensive experiments show that SRO achieves the highest average score in all evaluated target-model and agent-LLM combinations. In joint post-training across three capability domains and six benchmarks, SRO reaches 41.1 versus 15.4 for the strongest baseline on Qwen3-1.7B with GPT-5.4, and 53.8 versus 40.8 on Qwen3-4B with DeepSeek-V4-Flash. Our code is available at https://github.com/RuohLiuq/SRO.