Search papers, labs, and topics across Lattice.
This paper surveys 1,250 arXiv papers on recursive self-improvement (RSI) in AI, categorizing the systems based on what they improve and the degree of loop closure. It distinguishes between bounded self-refinement, which is already in industrial practice, and open-ended RSI, highlighting the challenges posed by grounding requirements and compute constraints. The authors propose a verification hierarchy for self-evaluation signals and identify critical failure modes, emphasizing the need for governance-grade measurement in this underexplored area of AI research.
Self-improvement in AI is not just a buzzword; it reveals a critical bottleneck in research direction-setting that keeps humans in the loop, highlighting the urgent need for better governance measures.
AI systems increasingly participate in their own improvement: revising their outputs, adapting their own harnesses during deployment, training on data they generate, and, increasingly, conducting AI research itself. This literature is described under a vocabulary ("self-refine," "self-reward," "self-play," "self-evolve") that conflates fundamentally different ambitions. We survey 1,250 arXiv papers (2024-2026) along two axes: what the system improves -- its behavior in deployment, its policy through training, its evaluator, or the research process itself -- and the degree of loop closure (human-in-the-loop to fully closed). The taxonomy separates bounded self-refinement -- convergent, evaluable, and already industrial practice -- from open-ended recursive self-improvement (RSI), which remains bounded by grounding requirements, collapse dynamics, and compute constraints on every measured axis. Its distinctive feature is a dedicated category for self-evaluation: every improvement loop is a claim that some signal can substitute for human judgment. We survey the evaluator design space -- judges, process reward models, verifiers, rubrics, meta-evaluation -- order the signals into a verification hierarchy from formal verifiers (strongest) to intrinsic self-assessment (weakest), and observe that demonstrated self-improvement strength tracks this hierarchy, that its failure modes (self-confirming loops, model collapse, diversity collapse) follow from its violations, and that the "research direction-setting" bottleneck keeping humans in the loop sits at the top of that hierarchy. We connect the technical literature to the theory of RSI limits and to the safety and governance questions raised by frontier-lab accounts of closing the loop, and identify governance-grade measurement of self-improvement as the field's most underpopulated niche.
