Search papers, labs, and topics across Lattice.
This paper introduces Maestro Order, a model-agnostic orchestration harness that enhances the reliability of problem-solving systems by composing various models through four structural primitives: decompose, ensemble, verify, and recurse, all managed by a budget-aware controller. The architecture allows for any model to function as a black-box solver while integrating a verifier ensemble that measures discrimination online, leading to significant improvements in reliability at a reduced computational cost. Results from Monte Carlo simulations demonstrate that verification can geometrically amplify reliability, achieving high levels of accuracy with minimal resource expenditure, while also identifying potential failure modes and providing actionable guidance for implementation.
Verification can geometrically amplify reliability in problem-solving systems, achieving up to 99.9% accuracy with strategic orchestration of models.
A single forward pass of a capable model is a fast, fluent, and unreliable problem-solver: it is right often enough to be useful and wrong often enough to be dangerous; in language models, such confident errors are known as hallucinations. We present Maestro Order, a model-agnostic orchestration harness that turns unreliable solvers into reliable problem-solving systems by composing them according to four structural primitives (decompose, ensemble, verify, and recurse) and a budget-aware controller that decides where to spend compute. The harness treats any model as a black-box base solver behind a uniform interface, layers a verifier ensemble whose discrimination is measured online, and allocates verification and voting to the stages with the highest marginal reliability per unit cost. We give the architecture, the message and state schema, the controller algorithm, and the engineering that makes it deterministic, observable, and fault-tolerant. We then specify an evaluation methodology (reliability at fixed cost, coverage, calibration, and ablations) and report results from a faithful Monte Carlo simulation of the harness over a parameterized solver/verifier model. The simulation reproduces the predicted laws quantitatively: verification amplifies reliability geometrically (e.g. $0.55\to0.98$ with two gates, $\to0.999$ with four), voting helps only above chance and is limited by shared errors, and a budget-aware controller reaches a target reliability at a small fraction of the cost of voting alone by selecting the cheapest mechanism for each regime. We close with failure modes (verifier gaming, correlated errors, and decomposition error compounding) and concrete guidance: build robust checkers, diversify solvers, and let the controller put compute where the information is.
