Search papers, labs, and topics across Lattice.
This paper introduces ReHarness, a novel recurrent harness framework for agentic continual learning that enables online adaptation to distribution shifts by incorporating backward revision of the harness state. Unlike existing methods that rely on forward-only information flow or external data, ReHarness uses the trajectory of each instance as feedback to revise past reflections, thereby propagating learning backward through the textual harness. Experiments across multiple benchmarks and LLM backbones demonstrate ReHarness's consistent superiority over strong baselines, showing improved stability, plasticity, and source-grounded revisions.
LLM agents can learn from past mistakes by retroactively revising their internal "harness" state, significantly improving continual learning performance.
Language model agents increasingly operate over long interaction streams, requiring online adaptation to distribution shifts through continual learning. However, existing agent harnesses typically either evolve their states only through forward information flow or rely on additional replay or validation data for harness optimization. To address this, we propose ReHarness, a recurrent harness framework for agentic continual learning that complements forward read and write with backward revision under a strictly online streaming protocol. We ask what information could have been captured differently in the harness state to better inform the current instance in the stream. Specifically, after each instance, we leverage the resulting trajectory as feedback to revise the reflections retrieved for that instance, thereby propagating feedback backward through the textual harness state. Across three continual learning benchmarks and three language model backbones, ReHarness consistently outperforms strong raw-context baselines, memory- and reflection-based methods, and prompt optimization approaches. Further analysis shows that ReHarness produces source-grounded and meaningful revisions, corrects overgeneralized or outdated experience under distribution shifts, and improves both stability and plasticity during continual learning.
