Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The authors formulate agent harness adaptation as meta-learning over executable programs, training an RL-based proposer model to iteratively revise an LM solver's control flow and tool scaffolding using execution feedback. This approach replaces parameter-space gradient updates with program-space edits, allowing agents to structurally adapt to novel environments at test time without model fine-tuning. Evaluated on reasoning and multi-hop QA, the proposer achieves out-of-domain transfer to unseen tasks, demonstrating that single-revision training enables sustained multi-round harness optimization.
Modifying executable agent harnesses rather than model weights allows test-time adaptation to generalize across unseen reasoning tasks without a single gradient step.
A language-model agent is jointly defined by its model and its harness, the executable program that organizes model calls, tool use, and information flow. Because different tasks call for different ways of organizing these operations, the harness needs to be adapted using feedback from the task at hand. We introduce harness learning, which trains a proposer model to revise a solver's harness using execution feedback. We formulate this process as meta-learning over executable programs, with harness revisions playing the role of weight updates in gradient-based adaptation. We train the proposer with reinforcement learning, using the task performance of revised harnesses as the reward. At test time, the proposer uses feedback from successive executions on a new task to refine the harness, without performing any parameter-space update. Experiments on reasoning and multi-hop question answering show that harness learning improves revision quality and that the ability to adapt at test time transfers to unseen tasks. Policies trained on individual revisions can continue improving harnesses over multiple rounds, while the benefits of training on revision sequences vary across settings. These findings suggest a path towards continually learning agents that turn accumulated experience into generalizable improvements.
CMU ML