Search papers, labs, and topics across Lattice.
This paper introduces EvoHarness-RL, a framework for training long-horizon LLM agents to learn and adapt their external execution support through offline harness policy learning. By utilizing a structured approach that incorporates Belief, Progress, and Experience (BPE) as policy-facing harness states, the agents can effectively manage state formation and runtime control, achieving a remarkable 96.9% success rate in task execution. The findings highlight two critical dynamics: harness annealing, which optimizes the frequency of harness calls, and harness evolution, which refines the external state into a more efficient, task-adaptive substrate.
Long-horizon LLM agents can achieve 96.9% task success by learning to adapt their external execution support through trainable harness policies.
Long-horizon LLM agents increasingly rely on external execution support to maintain state, track progress, invoke tools, verify outcomes, and reuse experience across interactions. However, effective harness use raises two coupled challenges: state formation from noisy interaction traces and runtime control over external-state access. Existing agents usually handle both through prompts, heuristics, or domain-specific conventions, leaving the external workspace and its usage policy manually engineered. To address this, we study the problem of harness policy learning, where agents learn harness policies offline and deploy them to construct and update external harness state online during runtime task execution. We introduce EvoHarness-RL, which exposes Belief, Progress, and Experience (BPE) as policy-facing harness state. Supervised harness fine-tuning teaches the base agent the harness action space and how to construct useful external state, while cost-aware GRPO explores coordination policies to selectively read, update, and consolidate that state during long-horizon interaction. Instantiated on ALFWorld with a Qwen3-8B LLM, EvoHarness-RL reaches 96.9% success and reveals two key dynamics: harness annealing, where training internalizes recurring harness-use patterns into the model policy and shifts the agent from frequent harness calls toward selective external-state access, and harness evolution, where progress updates and experience consolidation refine the harness into a compact, task-adaptive state substrate. These results suggest that long-horizon agents benefit from trainable policies for constructing and coordinating with external harness workspaces, beyond simply adding stronger tools or larger memories.
Meta AI