Search papers, labs, and topics across Lattice.
This paper introduces a unified black-box reinforcement learning (RL) framework that enhances the optimization of general agents through complex agent harnesses, addressing the challenges of scaling long-horizon tasks. By implementing a sandbox-based execution infrastructure and decoupling policy optimization from harness execution, the authors achieve significant performance improvements on the ClawGym-Bench and other challenging tasks. The results demonstrate that their approach can yield substantial gains in Pass@1 scores while maintaining stability across multiple optimization steps, thereby enabling effective training across diverse execution environments.
Black-box RL can boost agent performance by nearly 15 points on complex tasks, revealing a new frontier for scalable optimization.
Agent harnesses have substantially improved performance on long-horizon tasks by coordinating agent interactions with the environment. However, reinforcement learning through complex harnesses remains largely unexplored, as scaling such training to long-horizon agent tasks introduces fundamental challenges. In this work, we present a unified black-box RL framework for stable and scalable optimization of general agents through complex harnesses. Concretely, we first build a sandbox-based execution infrastructure that isolates task environments and harnesses within temporary sandboxes for large-scale concurrent rollouts. We then decouple policy optimization from opaque harness execution and place a serving proxy at the model boundary to capture model calls. To reconstruct multi-turn trajectories and improve training efficiency, we organize the captured calls into prefix trees and further adapt both critic-based PPO and critic-free GRPO to optimize over the recovered tree structure. Meanwhile, we maintain training-inference consistency throughout the optimization process. Finally, we introduce mix-harness training, allowing a single model to be jointly optimized by heterogeneous harnesses. With Qwen3-30A3B, black-box RL improves Pass@1 on ClawGym-Bench by 9.98 and 14.81 points through OpenClaw and Claude Code, respectively, while remaining stable over 200-400 optimization steps. Moreover, the framework yields consistent gains on more challenging tasks such as JobBench and OfficeQA. Overall, our framework enables effective, stable, and scalable optimization of general agents through black-box harnesses, supporting unified training across heterogeneous execution systems.
