Search papers, labs, and topics across Lattice.
This paper introduces HarnessLens, a budget-aware framework for the automated evolution of agent harnesses that enhances the efficiency of behavior-aware verification. By selectively verifying candidate modifications based on execution trajectories and behavior-relevant tasks, HarnessLens significantly reduces the evaluation budget while improving performance. The framework achieves an average performance increase of 7.6-13.6% across various benchmarks, showcasing its effectiveness in optimizing harness evolution under constrained conditions.
HarnessLens boosts agent performance by up to 13.6% while slashing evaluation costs through smarter, behavior-aware verification.
Agent harnesses shape how language-model agents use instructions, tools, and runtime components, but adapting these harnesses requires costly verification. Existing propose-and-verify methods typically score every candidate on a fixed task set, wasting rollouts on unrelated behaviors and allowing aggregate scores to obscure specific regressions. We introduce HarnessLens, a budget-aware framework for automated harness evolution. HarnessLens jointly explores the task space and user-configurable components, derives candidate modifications from execution trajectories, and selectively verifies each candidate on behavior-relevant tasks using an attributable-evidence gate. Across three agent harnesses and four benchmarks, HarnessLens improves average held-out performance by 7.6-13.6% while consuming substantially less evaluation budget than competing baselines. These results demonstrate that behavior-aware verification with explicit attribution enables more reliable and sample-efficient harness evolution under constrained interaction budgets. Our code is available at https://github.com/jhxu5214/HarnessLens.
