Search papers, labs, and topics across Lattice.
This paper introduces MiniOpt, a reinforcement learning framework that addresses the challenges of optimization generalization in large language models (LLMs) by employing a "reasoning-to-model-and-solve" approach. By utilizing a novel reward function, OptReward, which evaluates both the formulation and solution of optimization problems, MiniOpt enables effective policy learning without the need for expert demonstrations. Experimental results demonstrate that MiniOpt-3B achieves superior solving accuracy across various optimization tasks, outperforming other models with fewer than 10B parameters and maintaining competitive performance with larger models.
MiniOpt achieves the highest solving accuracy for compact models while requiring significantly fewer training resources than traditional methods.
Achieving strong optimization generalization across diverse optimization problems while requiring limited training resources remains a challenging problem for optimization-oriented large language models (LLMs). Existing approaches typically rely on large-scale supervised datasets, costly reasoning annotations, and expensive intermediate step verification, resulting in substantial training overhead. To address these challenges, we propose MiniOpt, a reinforcement learning framework that learns to solve optimization problems through an "reasoning-to-model-and-solve" paradigm. MiniOpt decomposes optimization reasoning into structured optimization modeling and executable solver generation. Building upon this paradigm, we introduce OptReward, a reward function with hierarchical score structure that jointly evaluates formulation and solution, enabling effective policy learning without expert demonstrations. We further develop an optimization-oriented policy optimization strategy that improves exploration efficiency and stabilizes reinforcement learning for compact models. Extensive experiments show that MiniOpt-3B exhibits strong optimization generalization across various optimization types, problem scenarios, and task domains. For models with fewer than 10B parameters, MiniOpt series achieves the highest average solving accuracy (SA). For models with more than 10B parameters, MiniOpt still shows competitive performance. These results suggest that optimization-oriented reward design and reinforcement learning provide an effective pathway for developing compact optimization-specialized language models with strong optimization generalization capabilities. The code is available at https://github.com/Hsiang-1/MiniOpt.
