Scalable Oversight & Alignment Theory - Weekly Roundup

Real-world professional tasks demand agents that learn from exploration and feedback, a capability unlocked by a novel engine that reconstructs environments to force agents through the full problem-solving lifecycle.

Predictive world models, not just static feedback, unlock substantial gains in recursive self-improvement across diverse domains, from code to scientific discovery.

Strict safety guarantees in RL are now achievable without performance trade-offs, even with neural approximations.

Frozen LLMs can autonomously achieve recursive self-improvement by learning and refining explicit, executable world models from interaction data.

Activations, not just descriptions, unlock the prediction of how LLM alignment generalizes to unseen values, revealing a structured value space.

Real-world professional tasks demand agents that learn from exploration and feedback, a capability unlocked by a novel engine that reconstructs environments to force agents through the full problem-solving lifecycle.

Predictive world models, not just static feedback, unlock substantial gains in recursive self-improvement across diverse domains, from code to scientific discovery.

Strict safety guarantees in RL are now achievable without performance trade-offs, even with neural approximations.

Frozen LLMs can autonomously achieve recursive self-improvement by learning and refining explicit, executable world models from interaction data.

Activations, not just descriptions, unlock the prediction of how LLM alignment generalizes to unseen values, revealing a structured value space.

LLM agents can now be monitored and intervened upon in real-time, saving 18% of compute by detecting failures within the first 8 steps with 83% accuracy.

Random features and projections unlock efficient quadratic entropy estimation for complex distributions and distances, revealing nuanced interdisciplinary research landscapes.

Interleaving source tasks with verification can save unboundedly many expected calls, but only if the verifier is exact; noisy verifiers flip this, demanding unbounded information and error.

Uncertainty in loss distributions cripples standard risk-averse ML; this work shows credal sets can reliably prevent catastrophic decisions without sacrificing average performance.

Self-improving AI systems can achieve stable learning dynamics and persistent utility advantages by strategically managing objective drift through co-evolutionary symbolic archives.

Language models can actively select replay targets to improve their own broad capability preservation, not just retain explicitly trained tasks.

LLM agents can learn from past mistakes by retroactively revising their internal "harness" state, significantly improving continual learning performance.

LLM agents can dynamically evolve their skill libraries, preventing obsolescence and improving long-horizon task performance by up to 7.8% through a fitness-driven lifecycle.

The $\gamma$-VC dimension, not just the standard VC dimension, dictates the sample complexity of boosting weak learners into strong predictors.

The hardest-to-distinguish competitor, not the average, dictates the speed of decentralized adaptive sensing.

Planning policy updates offline can significantly reduce the risk of performance regression while maximizing long-term gains, even with noisy improvement signals.

Even perfect AI agents fail to guarantee safety in realistic, time-sensitive, and partially observable environments, fundamentally challenging agent-agnostic alignment approaches.

Intrusion detectors confidently misclassify unseen attacks, but PairAudit's relational graph tokens uncover these overlooked errors, boosting human review accuracy by 30% on novel threats.

Frontier LLMs can achieve substantial recursive self-improvement, closing nearly half of the performance gap on complex benchmarks by jointly optimizing data, training, and execution harnesses within a unified, agent-native environment.

PPO-Clip's stationarity and critic tracking are jointly bounded by a common amplification factor dependent on algorithmic parameters, revealing a unified error mechanism.

A GP's ability to fit data is predictable from its input design alone, revealing potential contradictions before any responses are observed.

Jointly training path and flow networks with a novel alignment objective unlocks improved generative model performance by preventing path overfitting and low-entropy bottlenecks.

Robust bandit learning is tractable in a specific setting, but even minor deviations lead to NP-hardness, pinpointing a critical boundary for efficient unrealizable learning.

Conformal prediction set size reduction is formally justified as an information gain metric, bridging uncertainty quantification with classical information theory.

Frontier LLMs might be accumulating alignment debt faster than previously assumed, as fitting alignment scaling laws on smaller models can significantly underestimate the true burden at scale.

LLM agents achieve substantial performance gains by jointly learning credit at both the turn and token generation levels, overcoming limitations of existing single-level credit assignment methods.

Retraining on observed data in performative prediction can lead to unstable models, but a worst-case objective with label confidence intervals ensures convergence.

LLM tutors trained to adapt to individual student learning preferences achieve a 20.5pp performance boost and significantly outperform base models in human preference studies.

Tool-using agents frequently act without sufficient evidence, even when capable of strong static assessment, due to premature execution and incomplete investigation.

Embodied AI agents can now continuously self-improve by co-evolving their policies, training curricula, and judges, overcoming the bottleneck of static evaluation.

LLM agents achieve dramatic efficiency gains in long-horizon research by offloading conversational history to a central harness, enabling focused, role-specific contexts for each task.

Agentic loops can now reliably stop with provable guarantees on proposal quality, even with imperfect verifiers.

Web agents can now achieve state-of-the-art performance and robust test-time scaling by learning to self-verify their actions, bypassing the need for costly external judges during deployment.