Search papers, labs, and topics across Lattice.
Addressing the tendency of LLMs to make silent, invalid assumptions when converting incomplete natural-language prompts into operations research programs, the authors introduce the OR-Clarify benchmark and InterOPT, a dynamic pre-formulation clarification framework. Quantifying pre-formulation dialogue is critical for real-world optimization assistants, where missing business rules or constraints invalidate downstream mathematical programming. In experiments with simulated users, InterOPT substantially outperforms prior baselines in exact slot recovery under choice-based interaction and matches top methods in open-ended settings, successfully knowing when to ask questions versus when to stop.
Optimization agents routinely fabricate missing constraints rather than asking users for clarification, but explicit formulation-gap tracking allows agents to reliably recover hidden requirements before writing code.
Large language models (LLMs) are increasingly used to formulate optimization models from natural-language problem descriptions, yet realistic operations research (OR) requests are often incomplete: missing objectives, constraints, or business rules can change the resulting mathematical program. Existing evaluations largely assume a complete specification and therefore overlook whether an agent knows when clarification is needed before modeling. We introduce OR-Clarify, a benchmark for pre-formulation clarification. Each task presents a partial public problem description, withholds structured hidden slots, and evaluates agents through bounded interaction with a simulated user. The benchmark supports both openended and choice-based clarification, and measures slot recovery, stopping behavior, silent assumptions, and interaction cost. We further propose Interactive Optimization (InterOPT), a two-stage framework that identifies unresolved formulation-critical gaps and uses them to guide whether to ask the next question or to stop. In our choice-based experiments, InterOPT substantially outperforms all baselines in exact slot recovery; in the open-ended setting, it remains competitive with strong prior methods. Together, OR-Clarify and InterOPT reframe OR assistance as a selective completeness decision: clarify when needed, stop when ready, and quantify what remains missing.
