Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Hierarchical Self-Improvement (HSI), a framework that allows for the continuous evolution of task-specific harnesses for frozen LLMs, enabling them to adapt and improve through feedback without altering the underlying model. By implementing a structured approach where a task harness is rewritten based on environmental feedback, the authors demonstrate significant performance gains on various moderate-difficulty tasks, achieving improvements of up to 39.3% on BabyAI. The findings highlight the potential of harness evolution as a method for enhancing the capabilities of static LLMs, while also identifying the limitations imposed by the model's inherent capabilities.
Task-specific harness evolution can boost frozen LLM performance by over 39%, revealing a new path for enhancing AI agents without modifying their core architecture.
Modern LLM agents are often improved by modifying prompts, tools, or workflows manually, while the executable scaffold surrounding the model---the harness---is typically treated as a fixed artifact after deployment. This work studies an alternative where the harness is task-specific and continuously evolvable: each task family maintains its own harness, which is hot-swapped across iterations through a fixed task-injection seam and rewritten using environment feedback. We introduce Hierarchical Self-Improvement (HSI), a framework in which a single frozen LLM M operates across three hierarchical scopes: a task harness H that executes tasks, an evolver that rewrites H, and a meta-evolver that rewrites the evolver's strategy code under a frozen outer anchor. A thinking-on/off design isolates the contribution of harness evolution by disabling reasoning during task execution while enabling it during self-modification. HSI is bounded by two factors: a feedback-fidelity bound, since evolution requires informative reward signals to guide selection, and a backbone capability bound, since harness redesign cannot overcome limitations of the frozen model. On BALROG with DeepSeek-V4-Flash-Preview as the frozen backbone, HSI achieves consistent gains over the initial harness on moderate-difficulty tasks (+39.3 on BabyAI, +33.0 on Crafter, +25.0 on TextWorld, and +15.0 on MiniHack, all in raw \% Progress), while obtaining strong held-out generalization on BabaIsAI sub-suites (0.98 best-test on BreakStop and 1.00 on GoTo from a 20% unseen split). On tasks beyond the backbone's capability (NLE), harness evolution provides no improvement. These results demonstrate task-specific harness evolution as a viable axis for improving frozen LLM agents under clear empirical limits. Code is available at https://github.com/TailinZhou/hsi.