Scalable Oversight & Alignment Theory - Weekly Roundup

Agents trained with Agent-Reflex exhibit a 29.1% improvement on a noise-heavy benchmark, demonstrating robust failure recovery and proactive reflection even with faulty tool feedback.

Unexplained observations, not just supporting evidence, are the key to LLMs discovering truly novel hypotheses.

System One models can achieve direct, low-latency decision-making by treating answers as structured choices rather than serialized text, with Jeva demonstrating robust null detection and permutation invariance.

Explanations from Natural Language Autoencoders can identify their target activation but fail to discriminate it from plausible rivals, suggesting current training paradigms may not yield robust oversight tools.

A learned "Token-to-Go" progress meter enables multi-turn RL agents to effectively learn from sparse rewards, boosting performance by up to 7.2 points without increasing training costs.

Agents trained with Agent-Reflex exhibit a 29.1% improvement on a noise-heavy benchmark, demonstrating robust failure recovery and proactive reflection even with faulty tool feedback.

Unexplained observations, not just supporting evidence, are the key to LLMs discovering truly novel hypotheses.

System One models can achieve direct, low-latency decision-making by treating answers as structured choices rather than serialized text, with Jeva demonstrating robust null detection and permutation invariance.

Explanations from Natural Language Autoencoders can identify their target activation but fail to discriminate it from plausible rivals, suggesting current training paradigms may not yield robust oversight tools.

A learned "Token-to-Go" progress meter enables multi-turn RL agents to effectively learn from sparse rewards, boosting performance by up to 7.2 points without increasing training costs.

Personal agents mistakenly execute unacknowledged proposals as instructions, even with perfect user memory, due to a fundamental lack of "grounding awareness."

SDE Lyapunov estimates now directly translate to finite-time SGD convergence, simplifying analysis for novel optimizers.

Closed neighborhoods in planar graphs can be taught with at most two non-clashing examples, resolving a long-standing conjecture, yet distinguishing between one and two teaching examples is NP-hard.

Token-wise importance sampling in off-policy RLVR is now provably unbiased and practically superior, outperforming GRPO by up to 2.58% on math benchmarks.

LLM agents can learn to predict their own failure from the very first step of a task, achieving a 4.3x higher success rate in their most confident actions.

Multi-objective bandits can make finding the best arm *easier* than single-objective settings, even identifying larger optimal sets with fewer samples.

Few labels per question are sufficient for near-optimal recalibration when information is pooled across related tasks, with identification regret decaying exponentially with label count.

Representation reuse hinges not on source-task performance, but on preserving task-relevant distinctions and ensuring their consistent interpretability for downstream models.

AI agents can now autonomously rewrite their own code and workflows to improve performance, without retraining model weights.

Anchoring noise in the embedding space unlocks richer exploration for LLM RLVR, boosting solution coverage without altering model architecture or inference.

Agents trained on procedurally generated, harder environments significantly outperform those trained on original benchmarks, even with identical compute and optimization.

A single auditor can generate its own hard training examples, turning a static dataset into a renewable source for improved failure prediction in multi-agent systems.

Real-world professional tasks demand agents that learn from exploration and feedback, a capability unlocked by a novel engine that reconstructs environments to force agents through the full problem-solving lifecycle.

Predictive world models, not just static feedback, unlock substantial gains in recursive self-improvement across diverse domains, from code to scientific discovery.

Strict safety guarantees in RL are now achievable without performance trade-offs, even with neural approximations.

Frozen LLMs can autonomously achieve recursive self-improvement by learning and refining explicit, executable world models from interaction data.

Activations, not just descriptions, unlock the prediction of how LLM alignment generalizes to unseen values, revealing a structured value space.

LLM agents can now be monitored and intervened upon in real-time, saving 18% of compute by detecting failures within the first 8 steps with 83% accuracy.

Random features and projections unlock efficient quadratic entropy estimation for complex distributions and distances, revealing nuanced interdisciplinary research landscapes.

Interleaving source tasks with verification can save unboundedly many expected calls, but only if the verifier is exact; noisy verifiers flip this, demanding unbounded information and error.

Uncertainty in loss distributions cripples standard risk-averse ML; this work shows credal sets can reliably prevent catastrophic decisions without sacrificing average performance.

Self-improving AI systems can achieve stable learning dynamics and persistent utility advantages by strategically managing objective drift through co-evolutionary symbolic archives.

Language models can actively select replay targets to improve their own broad capability preservation, not just retain explicitly trained tasks.

LLM agents can learn from past mistakes by retroactively revising their internal "harness" state, significantly improving continual learning performance.

LLM agents can dynamically evolve their skill libraries, preventing obsolescence and improving long-horizon task performance by up to 7.8% through a fitness-driven lifecycle.

The $\gamma$-VC dimension, not just the standard VC dimension, dictates the sample complexity of boosting weak learners into strong predictors.

The hardest-to-distinguish competitor, not the average, dictates the speed of decentralized adaptive sensing.

Planning policy updates offline can significantly reduce the risk of performance regression while maximizing long-term gains, even with noisy improvement signals.

Even perfect AI agents fail to guarantee safety in realistic, time-sensitive, and partially observable environments, fundamentally challenging agent-agnostic alignment approaches.

Intrusion detectors confidently misclassify unseen attacks, but PairAudit's relational graph tokens uncover these overlooked errors, boosting human review accuracy by 30% on novel threats.

Frontier LLMs can achieve substantial recursive self-improvement, closing nearly half of the performance gap on complex benchmarks by jointly optimizing data, training, and execution harnesses within a unified, agent-native environment.

PPO-Clip's stationarity and critic tracking are jointly bounded by a common amplification factor dependent on algorithmic parameters, revealing a unified error mechanism.

A GP's ability to fit data is predictable from its input design alone, revealing potential contradictions before any responses are observed.

Jointly training path and flow networks with a novel alignment objective unlocks improved generative model performance by preventing path overfitting and low-entropy bottlenecks.

Robust bandit learning is tractable in a specific setting, but even minor deviations lead to NP-hardness, pinpointing a critical boundary for efficient unrealizable learning.

Conformal prediction set size reduction is formally justified as an information gain metric, bridging uncertainty quantification with classical information theory.

Frontier LLMs might be accumulating alignment debt faster than previously assumed, as fitting alignment scaling laws on smaller models can significantly underestimate the true burden at scale.

LLM agents achieve substantial performance gains by jointly learning credit at both the turn and token generation levels, overcoming limitations of existing single-level credit assignment methods.

Retraining on observed data in performative prediction can lead to unstable models, but a worst-case objective with label confidence intervals ensures convergence.

LLM tutors trained to adapt to individual student learning preferences achieve a 20.5pp performance boost and significantly outperform base models in human preference studies.

Tool-using agents frequently act without sufficient evidence, even when capable of strong static assessment, due to premature execution and incomplete investigation.

Embodied AI agents can now continuously self-improve by co-evolving their policies, training curricula, and judges, overcoming the bottleneck of static evaluation.

LLM agents achieve dramatic efficiency gains in long-horizon research by offloading conversational history to a central harness, enabling focused, role-specific contexts for each task.