CMU Machine Learning

Faster computer-use agents are bottlenecked by their speed, but existing benchmarks are irreproducible; this new standardized framework reveals counterintuitive speed-efficiency trade-offs and that current open-weight models lag behind.

Autonomous agents can recursively self-improve LLM-generated games, achieving superhuman performance and drastically cutting generation costs by internalizing development experience.

Long-horizon agents plateau under naive inference scaling unless compute is explicitly redirected from task workers to a dedicated meta-reasoning controller that manages execution flow.

Modifying executable agent harnesses rather than model weights allows test-time adaptation to generalize across unseen reasoning tasks without a single gradient step.

Autonomous model post-training can outperform human-crafted instruct baselines on hard coding benchmarks while completely eliminating reward hacking once experimental exploration is governed by a sandboxed OS and a dynamic peer-review DAG.

Several open research questions are identified, including how to efficiently record incidents and how to determine whether vulnerabilities and incidents generalize, and privacy requirements are summarized and research directions for the secure and trustworthy deployment of AI agents are outlined.

This work formalizes poison selection as oracle-budgeted set optimization and introduces SAILS (Set-level Audit-Informed Iterative Learned Selection), which learns a set scorer from a few hundred finetune-and-evaluate runs, ranks millions of candidate sets, and audits only a small shortlist.

DCP provides a common evidence language for useful outcomes, alternative routes, and feedback effects across AI research, and is applied to Core, recovered, and audit-incomplete decisions.

Personalized AI agents can achieve up to 20.9% better task success by learning from user feedback in real-time, reshaping the landscape of human-AI collaboration.

Agents fall short in machine learning development, often locked in narrow loops while humans adapt and innovate across tasks.

WeClawArena reveals how cross-user agent collaboration can be systematically audited, exposing vulnerabilities in personal workspaces that could compromise user privacy and security.

Agentic Commerce World reveals that process-level evidence is crucial for accurately evaluating AI agents in dynamic market environments, challenging traditional reliance on final outcomes alone.

Real-time manipulation policies can now react to sensory input every 40ms, achieving up to a 30% boost in success rates in dynamic environments.

TALRanker autonomously navigates the trade-off between efficiency and accuracy in document reranking, achieving state-of-the-art results without the latency of excessive tool calls.

Agents can now work independently on data changes while humans maintain oversight, revolutionizing collaborative data management.

GaP outperforms traditional methods in variational automation tasks by leveraging directed computation graphs for real-time adaptability and improved success rates.

PACE-Bench predicts agentic performance with remarkable accuracy while slashing evaluation costs to a fraction of traditional methods.

Even top-performing AI models struggle with PowerPoint tasks, achieving only 45% success rates despite a robust evaluation framework that rewards nuanced performance.

FARS challenges the boundaries of automated research by producing 166 papers across 67 topics, revealing both its potential and pitfalls in AI-driven science.

ANTAP achieves near-zero vulnerability to description-based attacks, fundamentally transforming how agents are evaluated and routed in multi-agent systems.

Grasp datasets can revolutionize robotic dexterity, enabling significant improvements in articulated tool use performance.

Agents should enhance causal discovery workflows without compromising the integrity of causal claims by relying solely on data and expert knowledge.

Co-authorship with humans can significantly enhance merge rates for certain AI coding agents, but this effect vanishes when accounting for repository selection and PR structure.

Agents can now share and reuse knowledge, cutting down task execution time and improving performance without the need for coordination or joint training.

Phoenix resolves GitHub issues with 75% accuracy while ensuring safety through a multi-agent system, but still faces challenges in planner localization.

Diminishing returns in parallel sampling can be overcome by generating diverse initial queries, leading to substantial performance gains in multi-hop question answering.

Claude Opus 4.6 outperforms its peers, solving over half of the complex tasks in a personalized desktop environment, revealing critical gaps in current AI capabilities.

Recycling zero-variance queries during training can boost model performance, enabling a smaller 1.7B parameter model to match or exceed the accuracy of larger counterparts.

iOSWorld reveals that even state-of-the-art models falter in multi-app reasoning, achieving only 37% accuracy, underscoring the complexity of personal context in AI interactions.

ALMANAC reveals that agents can significantly improve their collaborative competence by learning from detailed human mental model annotations.

Closed-loop memory optimization can boost software engineering agents' success rates by over 5% while slashing computational costs by nearly 10%.

Multi-agent coordination can boost task completion efficiency by up to 25.5%, transforming how we approach complex computer use challenges.

Leading LLMs falter in Korean web-browsing tasks, achieving less than half the accuracy found in previous benchmarks.

Even the top-performing LLM struggles with realistic user interactions, achieving only 61% success in complex task scenarios.

Over 54% of actions taken by leading LLM coding agents in realistic projects result in harmful safety violations, exposing critical gaps in current safety alignment.

LLMs can now automatically debug Prolog code, offering personalized feedback at scale for logic programming education.

LLMs can ace prediction tasks in causal environments, but still fail to grasp the underlying causal mechanisms, revealing a critical blind spot in their reasoning abilities.

RocketSmith achieves a remarkable 80% accuracy in predicted rocket apogee, showcasing the power of agentic systems in aerospace design.

Stop blindly trusting synthetic data for agent evaluation: SynAE reveals that no single metric can fully capture its quality, demanding a multi-faceted approach.

Achieve SOTA reasoning performance with a 30B parameter model using up to 95% fewer tokens by explicitly controlling when and how deeply an LLM plans.

LLMs may only account for 11-26% of high-level goal-setting in collaborations, but they exert far more influence by shaping the micro-decisions and concrete requirements that define those goals.

LLMs can now design better computer architectures than humans, but only if you give them the right starting point.

Today's best web agents are shockingly inefficient, achieving only 1.15% trajectory efficiency on realistic long-horizon tasks, revealing a critical need to move beyond simple success rates.

Continual learning for LLM agents hits a wall: scaling models doesn't reliably improve skill generation, and self-feedback can lead to recursive drift.

Forget carrots and sticks: contracts and mediation are the surprisingly effective keys to unlocking cooperation between LLMs, even when individual incentives push toward defection.

Stop wasting tokens on irrelevant questions: reward models that ask about task relevance and user answerability can slash question count by 41% while matching GPT-5's issue resolution rate.

Democratizing human-AI interaction research, CoGrid and MUG offer accessible tooling for deploying web-based multi-agent experiments.