Search papers, labs, and topics across Lattice.
MIRA-Math introduces a novel benchmark designed to evaluate the capability of models to solve mathematical problems while requesting missing information under strict constraints. By focusing on scenarios where the solver must identify and request a single necessary fact to complete a problem, this benchmark allows for a clearer assessment of the interplay between information requesting and mathematical reasoning. Experiments reveal that models can successfully request relevant facts but often struggle with the subsequent computation, highlighting a disconnect between information retrieval and reasoning accuracy.
Models can ask for the right mathematical fact but still fail to compute the correct answer, revealing a critical gap in reasoning capabilities.
Mathematical reasoning benchmarks typically provide all facts needed to solve each problem, while interactive benchmarks often mix reasoning with tools, retrieval, and long-horizon dialogue. We introduce MIRA-Math, a benchmark for a narrower diagnostic capability: solving mathematical problems whose full latent state has a unique answer, but whose solver-facing view is missing exactly one necessary atomic fact. The solver must request the missing information in natural language under a strict budget and then integrate the returned fact into an exact final answer. A fixed constrained LLM responder sees only the dataset-provided atomic fact and must either offer the quoted fact when the request matches it, or decline otherwise. Thus, instance generation, typed hint specifications, validation, and final-answer verification are deterministic, while request metrics are measured under a fixed LLM-mediated responder channel. MIRA-Math contains 2{,}310 generated instances from 22 typed mathematical families spanning algebra, probability, linear systems, discrete structures, signal processing, Markov chains, circuits, interpolation, and numerical boundary-value problems. Experiments across frontier and small models show that request success and final-answer accuracy are separable: models may ask for the right fact yet fail the downstream computation, or fail before obtaining the canonical hint. We release generators, verifiers, prompts, run metadata, and dataset documentation to support reproducible evaluation of minimal information requesting in mathematical reasoning.
