Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces the Harness-of-Harness (HoH) framework, which enables LLM-based coding agents to autonomously develop software through iterative planning, coding, and testing loops. By balancing repair with capability growth and emphasizing small, verifiable increments, HoH significantly enhances the performance of coding agents, achieving an average relative gain of 52.25% across various benchmarks. In a multi-day deployment, HoH successfully developed a fully functional first-person-shooter game, showcasing its ability to produce coherent and polished software without human intervention.
Coding agents using the Harness-of-Harness framework can achieve over 52% performance improvement in autonomous software development, revolutionizing how we approach LLM-based coding tasks.
This paper studies autonomous software development, in which LLM-based coding agents transform high-level requirements into complete, functional, and usable software systems without human intervention. We introduce Harness-of-Harness (HoH), a framework that enables coding agents to continually improve software during autonomous development. HoH operates on existing coding-agent harnesses, and organizes their executions into iterative planning-coding-testing loops. To sustain improvement across loops, HoH balances repair with capability growth, scopes development into small and verifiable increments, separates implementation-time testing from independent evaluation, and constrains verifiable outputs rather than prescribing agent workflows. It progressively exposes deliverables, role-specific tools, and skills, encourages reuse rather than recreation, and maintains versioned project histories. On GameCraft-Bench, FrontierSWE, and ProgramBench, three harness-model pairs (Codex with GPT-5.5, OpenCode with DeepSeek-V4-Pro, and Pi with MiniMax-M3), HoH consistently outperforms the corresponding standalone harnesses, achieving an average relative gain of 52.25 percent and a maximum gain of 82.86 percent after three iterations. In a multi-day deployment with more than 70 iterations, HoH autonomously develops a first-person-shooter game, featuring a coherent storyline, fully implemented core mechanics, human-playable experience, polished visuals and integrated audio. Github: https://github.com/Flesymeb/HarnessOfHarness Project Page: https://flesymeb.github.io/HarnessOfHarness/