Search papers, labs, and topics across Lattice.
StarHarness is a novel framework designed to evolve environment-specific agent harnesses while keeping model weights fixed, focusing on enhancing performance in enterprise settings. By stratifying tasks based on baseline failure behavior and separating search tasks, the framework achieves a remarkable 20-35 percentage point improvement in benchmark performance across various environments after just a few accepted changes. Notably, these enhancements not only persist on previously excluded tasks but also transfer effectively across different model families, indicating a robust solution to model-environment mismatches.
Evolving agent harnesses can yield a 20-35% performance boost in enterprise environments without altering model weights.
We present StarHarness, a framework for evolving environment-specific agent harnesses while keeping model weights fixed. The evolved harness can include prompt and task framing, tool interfaces, skills, MCP-backed providers, subagent structure, and agent-loop configuration. StarHarness constructs a compact evolution pool by stratifying tasks according to baseline failure behavior, separates proposer-visible search tasks from proposer-hidden selection tasks, and reserves held-out tasks for evaluating generalization. Across ITBench SRE, EnterpriseOps-Gym ITSM, and AutomationBench Finance, harness evolution improves full-benchmark performance by 20-35 percentage points over the default harness after 4-12 accepted changes per environment. These gains persist on tasks excluded from evolution and transfer without re-evolution across GPT and Qwen model families. Trace analysis links the improvements to interface repairs, environment conventions, and operational knowledge that compresses search, with fewer false-positive diagnoses and shorter trajectories in several settings. StarHarness therefore offers a practical way to reduce persistent model-environment mismatch in tool-rich enterprise tasks.
Mila