Tool Use & Agents - Weekly Roundup

Coding agents routinely fail to build agents that truly meet user requirements, with the gap between demonstration and delivery being the norm.

Real-world professional tasks demand agents that learn from exploration and feedback, a capability unlocked by a novel engine that reconstructs environments to force agents through the full problem-solving lifecycle.

Human GUI interactions can be precisely captured and translated into reusable agent skills, dramatically boosting performance on unseen software and workflows.

Predictive world models, not just static feedback, unlock substantial gains in recursive self-improvement across diverse domains, from code to scientific discovery.

Existing 3D asset generators can now produce objects that not only look good but also move and function realistically, passing physics tests for stability and collision-free articulation.

Coding agents routinely fail to build agents that truly meet user requirements, with the gap between demonstration and delivery being the norm.

Real-world professional tasks demand agents that learn from exploration and feedback, a capability unlocked by a novel engine that reconstructs environments to force agents through the full problem-solving lifecycle.

Human GUI interactions can be precisely captured and translated into reusable agent skills, dramatically boosting performance on unseen software and workflows.

Predictive world models, not just static feedback, unlock substantial gains in recursive self-improvement across diverse domains, from code to scientific discovery.

Existing 3D asset generators can now produce objects that not only look good but also move and function realistically, passing physics tests for stability and collision-free articulation.

Frozen VLAs can recover from unseen actuator faults by inferring joint capabilities online, boosting success rates from 24.8% to 59.3% on 28 tasks.

Frozen LLMs can autonomously achieve recursive self-improvement by learning and refining explicit, executable world models from interaction data.

LLM agents significantly outperform numerical optimizers in four out of five black-box optimization domains, but adding more numerical tools doesn't always help.

Pretrained LLMs can drive complex robot navigation tasks across diverse environments and requests without any navigation-specific fine-tuning, by offloading motion execution to specialized tools.

A single visual world model can now execute complex procedural tasks end-to-end, adapting its generation to the actual outcome of each predicted action.

Spatial coding agents can be distilled into standalone MLLMs, achieving superior spatial reasoning and generalization without external tools.

LEGO-building AI agents can assemble physically valid structures from text prompts, but their designs lag significantly behind human creativity and complexity.

LLM agents can now be monitored and intervened upon in real-time, saving 18% of compute by detecting failures within the first 8 steps with 83% accuracy.

Visual-native skill cards, directly accessible to VLMs, unlock a self-reinforcing loop of policy improvement and skill accumulation, drastically boosting sample efficiency.

Ambiguous object selection in text-guided image editing is solved by directly annotating the canvas, achieving state-of-the-art performance in complex edits.

LLMs can now autonomously learn to self-improve at test-time, nearly doubling utility by intelligently selecting and applying TTT strategies.

Manual assembly instruction writing is now obsolete: LLMs can generate accurate, CAD-grounded procedures, preserving engineering details and passing automated checks.

A frozen video tokenizer's latent can be factored into controllable and reward-relevant components, unlocking significant downstream planning gains.

Casual images are now enough to generate high-fidelity, scene-aligned 3D objects with accurate pose, even from sparse observations.

Human motion alone, when translated into a shared latent space, unlocks zero-shot generalist humanoid control for locomotion and manipulation tasks.

Research reports can now be continuously updated with over 60% fewer search calls and 33% less token consumption, maintaining high quality over time.

LLMs can now generate executable feature programs from long text, enabling efficient structured data prediction without real-time LLM inference.

LLMs adapt neural operator configurations using both a strong task-dependent prior and real-time experimental feedback, outperforming Bayesian optimization.

A residual memory mechanism allows LLMs to retain crucial context for long-horizon tasks, achieving near full-context performance with only 5.2% of input positions.

LLMs can be steered to reliably interpret complex genomic variants by grounding their reasoning in deterministic biological computations and evidence verification, avoiding hallucinated TF binding changes and fabricated experimental support.

Agents learning to optimize local rewards in spatial public goods games can inadvertently create detrimental spatial patterns that reduce collective welfare by over 40% compared to random movement.

Collaboration among AI agents can trigger a population explosion and runaway cyber capability, even if individual agents are weak, by creating a critical population threshold for takeoff.

Most AI agent skills copied on GitHub originate from a few hidden hubs, rendering security patches ineffective for the vast majority of users.

Fine-grained embodied interactions between multiple agents can be effectively modeled by synchronizing ego-stream generation with cross-view action consistency and shared environment memory.

LLM agents can achieve high task accuracy while exhibiting profound epistemic *im*humility, failing to acknowledge uncertainty even when presented with contradictory evidence.

A unified multimodal agent, OneSearch-VL, achieves state-of-the-art performance on diverse visual research tasks by explicitly modeling and preserving interdependencies within a Visually Grounded Evidence Graph.

Coding agent benchmarks can now dynamically scale difficulty and avoid data leakage by having agents evaluate each other, with the latest models generating tasks that defeat older solvers 10x more often.

Agents can learn explicit, reusable mental models of partners to dramatically improve multi-agent interaction quality and Theory-of-Mind capabilities.

AI agents can learn to climb leaderboards in complex adversarial games by refining executable policies, but struggle with intricate rule sets

Synthesized software environments unlock a 50% boost in CUA performance on unseen benchmarks, proving scalable supervision is key.

LLM agents can learn from past mistakes by retroactively revising their internal "harness" state, significantly improving continual learning performance.

Amortized agentic policy discovery enables personalized test-time scaling to satisfy complex, multidimensional user requirements, outperforming single-objective optimization on unseen profiles.

A novel contrastive feedback mechanism allows LLM agents to learn skill proposals that are aligned with the current actor's behavior without requiring expensive skill evaluations.

LLM agents can dynamically evolve their skill libraries, preventing obsolescence and improving long-horizon task performance by up to 7.8% through a fitness-driven lifecycle.

LLM agents can now reliably track and fulfill complex deliverable requirements across numerous files without losing context, thanks to an environment-side "tab" that acts as a persistent, shared task ledger.

AI agents are now building climate models that outperform published research and offer novel hypotheses for long-standing scientific debates.

An LLM-based agent achieves 18.3% out-of-sample returns in stock options trading by learning to select structured strategies, but surprisingly, retaining news during RL training halves performance.

Frontier LLMs can achieve substantial recursive self-improvement, closing nearly half of the performance gap on complex benchmarks by jointly optimizing data, training, and execution harnesses within a unified, agent-native environment.

Uncoded crash factors like phone use and medical episodes are vastly undercounted in police records, with narratives revealing 2x more injury crashes than coded fields.

A frozen graph backbone effectively preserves past structural knowledge in continual multi-agent reinforcement learning, dramatically reducing forgetting and boosting transfer across tasks with distinct graph structures.

Planning in visual world models can be accelerated by $5\times$ by pruning $50\%$ of spatial tokens based on their influence on the planning cost gradient, without sacrificing performance on most control tasks.

AI is poised to revolutionize ARPES, moving beyond data analysis to enable fully autonomous, self-driving experimental labs.

Intermediate retrieval signals, not just final outcomes, unlock efficient credit assignment for training LLM search agents.

AI agents can now build and reuse persistent, artifact-specific knowledge, dramatically improving their ability to find relevant information for complex tasks and reducing operational costs.

LLMs can learn to actively compute evidence by routing through retrieval and computation, dramatically improving performance on complex, multi-source reasoning tasks.

Physical robot learning can now autonomously discover and exploit informative experiments to continuously improve foundation models, achieving 77% success on RoboCasa365 and zero-shot transfer to real hardware.

Frontier multimodal agents struggle to autonomously acquire and utilize task-relevant information in unfamiliar physical environments, failing 77% of the time on a new exploration benchmark.

Key contribution not extracted.

Real-world coding agent usage reveals that task flow, not just task completion, dramatically impacts agent cost and effectiveness, suggesting current benchmarks are misaligned with user behavior.

Trillion-parameter agentic RL refits are now 12-40x faster, shrinking an 87.5-minute full checkpoint transfer to 150 seconds by sending only bit-exact weight deltas.

A single, lightweight model-internal mechanism can effectively bridge the gap between corpus retrieval and long-context inference, outperforming specialized SOTA systems on both.

Storing knowledge directly in LLM parameters offers a flexible, efficient alternative to in-context learning, bypassing context length limitations and repeated encoding costs.

Agents can learn to act reliably in new environments by generating their own practice tasks and distilling reusable heuristics from self-identified failures, achieving significant performance gains without human supervision or pre-defined benchmarks.

Personal LLM agents can significantly improve recommendation quality by mediating platform rankings with cross-platform history, but naive mediation introduces substantial harmful overrides.

Embodied agents can now reliably achieve complex, multi-stage goals in dynamic environments by learning to predict and ground targets from decoupled visual context, even when the goal is out of view.

Latency in voice assistants plummets by 20% by speculatively executing tool calls *during* speech recognition, not after.

Sequential recommenders can dramatically boost catalog diversity by adaptively pruning action sets, achieving up to 5.21x diversity gains without sacrificing session depth.

Context compaction in long-horizon agents is surprisingly insensitive to recent agent behavior, with simple "has-written" heuristics measuring trajectory phase rather than predictive power.

Parallel LLM agents can finally achieve near-linear speedups by strategically parallelizing only when token-based cost estimates predict a net gain, eliminating redundant work and reconciliation overhead.

LLM agents can autonomously optimize RAG pipelines by diagnosing retrieval vs. generation failures, outperforming traditional hyperparameter tuning and achieving Pareto efficiency.

LLM agents achieve substantial performance gains by jointly learning credit at both the turn and token generation levels, overcoming limitations of existing single-level credit assignment methods.

LLM-generated heuristics can now guarantee optimal plans, outperforming specialized planners by combining generative power with formal verification.

LLMs can decode partner intents but struggle to translate this knowledge into cooperative actions, with explicit action recommendations proving far more effective than abstract rule statements.

LLM agents struggle to translate general capabilities into cost-efficient, reusable solutions, often failing to outperform simple distillation baselines despite significant zero-shot prowess.

Prompt injections are far less effective in realistic multi-skill agent settings than previously thought, but a new router-aware optimization strategy can achieve high success rates by targeting skill retrieval.

Tool-using agents frequently act without sufficient evidence, even when capable of strong static assessment, due to premature execution and incomplete investigation.

Embodied AI agents can now continuously self-improve by co-evolving their policies, training curricula, and judges, overcoming the bottleneck of static evaluation.

Web agents trained with adaptive adversarial simulation achieve 33.6% higher completion rates against unseen prompt injection attacks, even outperforming frontier adversaries.

Discovering molecules with rare properties can now be an order of magnitude faster by testing complex mixtures and deconvoluting activity, slashing experimental costs from $\mathcal{O}(d)$ to $\mathcal{O}(\log d)$ or $\mathcal{O}(1)$.

LLM agents achieve dramatic efficiency gains in long-horizon research by offloading conversational history to a central harness, enabling focused, role-specific contexts for each task.

AI agents in a simulated scientific ecosystem rediscovered over 60% of key findings from recent research papers, suggesting a viable path to autonomous open-ended scientific discovery.

LLMs can now autonomously debug complex chip designs by translating natural language queries into SQL against massive simulation dumps.

Semantic decision engines routinely fail network deadlines and select infeasible actions, with most claims of real-time performance unsupported by evidence.

LLM agents can now dynamically curate multimodal memory on-demand, achieving optimal performance-cost-latency trade-offs by intelligently choosing between pre-computed and runtime-generated evidence.

LLM agents in decentralized marketplaces are demonstrably exploitable, with adversarial prompting enabling over 50% fraudulent transaction completion and doubling agent profits.

Learned corner selection for cooperative guards dramatically boosts deployment efficiency and agent utilization in partially observable environments, outperforming established methods without sacrificing formal guarantees.

Local AI models now predict user well-being from passive computer activity, offering explainable feedback directly within a workplace sensing application.

Network traffic alone can reveal LLM agent risks and granular behaviors, but current analysis methods fail to distinguish subtle malicious activities or semantically similar scenarios.

Hierarchical procedural understanding unlocks a 9.6%p performance boost in proactive AI assistance, enabling adaptive guidance that outperforms in-context learning by 57.1%p.

Agentic loops can now reliably stop with provable guarantees on proposal quality, even with imperfect verifiers.

Prompt injection defenses can degrade LLM agent utility by altering benign behavior; RAISED achieves robustness without this cost through self-distillation on generated scenarios.

Flowcharts break silently across aspect ratios, but a new agentic pipeline with VLM critics now guarantees connectivity and editability.

Robots can now learn complex manipulation tasks from a single demonstration video by recursively accessing fine-grained sub-event details, boosting success rates by up to 9%.

Web agents can now achieve state-of-the-art performance and robust test-time scaling by learning to self-verify their actions, bypassing the need for costly external judges during deployment.

LLMs forget user intent after a rejected suggestion, treating it as active even when superseded, leading to task failure.