Search papers, labs, and topics across Lattice.
Meta-Harness is introduced as an outer-loop optimization system that searches over harness code for LLM applications, leveraging an agentic proposer with access to source code, scores, and execution traces of prior candidates via a filesystem. It achieves significant performance gains, including a 7.7-point improvement in online text classification with 4x fewer context tokens and a 4.7-point accuracy increase in retrieval-augmented math reasoning across held-out models. The system also surpasses hand-engineered baselines in agentic coding on TerminalBench-2, demonstrating the effectiveness of automated harness engineering through richer access to prior experience.
Stop hand-coding your LLM harnesses: Meta-Harness can automatically discover harnesses that outperform state-of-the-art systems while using fewer context tokens and generalizing across models.
The performance of large language model (LLM) systems depends not only on model weights, but also on their harness: the code that determines what information to store, retrieve, and present to the model. Yet harnesses are still designed largely by hand, and existing text optimizers are poorly matched to this setting because they compress feedback too aggressively. We introduce Meta-Harness, an outer-loop system that searches over harness code for LLM applications. It uses an agentic proposer that accesses the source code, scores, and execution traces of all prior candidates through a filesystem. On online text classification, Meta-Harness improves over a state-of-the-art context management system by 7.7 points while using 4x fewer context tokens. On retrieval-augmented math reasoning, a single discovered harness improves accuracy on 200 IMO-level problems by 4.7 points on average across five held-out models. On agentic coding, discovered harnesses surpass the best hand-engineered baselines on TerminalBench-2. Together, these results show that richer access to prior experience can enable automated harness engineering.
Stanford HAI