Tool Use & Agents - Weekly Roundup

DataMagic achieves over 95% success rate and an 83% quality improvement in data video generation by declaratively orchestrating multi-agent scene creation, surpassing even state-of-the-art LLMs.

Ultra-long video grounding is solved by evolving reusable agent skills and tools from VLM reasoning, bypassing parameter updates and separate planning models.

A single capable verifier can dramatically improve terminal agent action reliability, achieving higher success rates at lower costs than simply generating more trajectories.

Learned world models can achieve real-time control in live games, outperforming behavioral cloning and maintaining high-frequency action loops.

Most failed multi-turn agent trajectories contain a single, recoverable pivotal mistake that, if addressed, can restore task success, a phenomenon PivotOPD exploits for significant performance gains.

Ultra-long video grounding is solved by evolving reusable agent skills and tools from VLM reasoning, bypassing parameter updates and separate planning models.

A single capable verifier can dramatically improve terminal agent action reliability, achieving higher success rates at lower costs than simply generating more trajectories.

Learned world models can achieve real-time control in live games, outperforming behavioral cloning and maintaining high-frequency action loops.

Most failed multi-turn agent trajectories contain a single, recoverable pivotal mistake that, if addressed, can restore task success, a phenomenon PivotOPD exploits for significant performance gains.

State-of-the-art VLMs struggle to execute scientific research workflows, revealing a critical gap in their ability to interact with specialized software and produce verifiable results.

LLMs can overcome knowledge gaps in evolutionary search by dynamically co-evolving solutions and targeted web queries, leading to substantial gains in scientific discovery.

Robots can now learn from physical failures in real-time, autonomously revising their own capabilities to achieve unprecedented long-horizon manipulation success.

Faster computer-use agents are bottlenecked by their speed, but existing benchmarks are irreproducible; this new standardized framework reveals counterintuitive speed-efficiency trade-offs and that current open-weight models lag behind.

LLM agents can improve their decision-making by learning from their own diagnosed failures, with first-proposal corrections boosting success rates by over 30 percentage points.

A listwise agent inspecting top candidates holistically boosts patch retrieval accuracy by over 27% compared to treating them independently.

Standard information retrieval methods can match LLM-mediated skill selection for agents, slashing per-task costs from $51 to $27.

Visual perception agents can achieve significant performance gains by actively seeking information beyond a single glance, outperforming static models by up to 30 points on challenging benchmarks.

Imagined failures in a world model can pinpoint exactly which robot skill components need new demonstrations, boosting success from 13% to 75% with minimal data.

LLM agents trained solely in rule-generated fictional worlds achieve state-of-the-art transfer to real-world search tasks, outperforming agents trained on actual data.

A single graph edge misspecification can transform a seemingly robust causal action verifier into a dangerous liability, executing harmful actions with internally valid but misleading certificates.

AutoResearch systems can achieve up to 28% fewer errors by making competing behaviors visible and actively steering edits to resolve their inherent conflicts.

Lightweight sentinels can learn to proactively intervene in coding agent actions, boosting task success by up to 14% by preventing costly error propagation.

Parallel search agents can now achieve higher accuracy than generative aggregators by leveraging TRACE to intelligently select the best trajectory from aggregated cross-rollout evidence.

Autonomous agents can recursively self-improve LLM-generated games, achieving superhuman performance and drastically cutting generation costs by internalizing development experience.

Frontier LLMs can be trained to selectively ignore unreliable external guidance, a capability crucial for robust agentic behavior beyond simple task accuracy.

Agentic discovery yields a deletion-only DNA designer that outperforms greedy and random approaches, achieving substantial sequence reduction while preserving predicted function.

Most multimodal models fail to truly understand tool use in real-world egocentric videos, with even Gemini-3.1-Pro struggling to ground actions in visual evidence.

LLM agents can now adapt their action spaces dynamically from natural language, bypassing the need for pre-defined finite fields and accelerating inference through compositional policies.

Clinical LLM agents can now autonomously gather evidence from longitudinal medical records and imaging, achieving superior accuracy to GPT-5.4 and Claude Opus.

LLM agents abandon their own declared plans up to 78% of the time, but enforcing faithful execution via pattern-specific scaffolding nearly doubles benchmark success rates.

Teaching agents *what* unstated information to hunt down—rather than just *when* to act—enables small models to beat 15× larger baselines at multi-hop task completion while asking fewer questions.

LLM agents waste immense token budgets repeatedly rediscovering identical cross-document links on every query—pre-indexing corpora into an offline entity-document graph solves fragmented multi-hop retrieval while actually lowering token costs.

Language models that edit their own context window like a mutable file beat external agent scaffolding across long-horizon tasks while cutting compute by up to 59%.

Heterogeneous visual agents can boost downstream performance by up to 4.9 points across diverse tasks simply by reading from a shared, self-organizing 2B memory tree—without updating a single host model weight.

Computer-use agents can bypass brittle, hand-engineered APIs entirely by learning when to substitute mouse clicks with native shell commands, yielding a 14.8-point leap on OSWorld.

Robots no longer need human intervention to learn from their mistakes: foundation-model-guided exploration over modular skills can autonomously synthesize corrective rollouts that distill directly into low-level policy weights.

Translating learned strategic principles into an external runtime harness yields a 12-point performance boost over directly feeding those same principles into an agent's context window.

Frozen tabular foundation models can dominate competitive ML benchmarks without parameter updates if an LLM agent is tasked strictly with evolving the semantic data pipeline around them.

Today's frontier agents virtually collapse in professional engineering environments, solving just 3.6% of multi-software workflows when held to real-world physical and geometric constraints.

Training a small advisor to steer frozen frontier LLMs actually degrades if it learns from every post-hoc correction—selectively distilling only the advice that causally shifts execution yields up to a 6.4% gain and robust cross-family model transfer.

Append-only memory guarantees attention collapse in long-horizon video reasoning, but actively rewriting a bounded working memory from an external filesystem boosts hard-subset task accuracy by over 20%.

Self-improving coding agents do not require downstream reward signals to evolve: bootstrapping purely from historical self-modification traces matches the performance of benchmark-guided search at a fraction of the cost.

A 220× acceleration of LTL-guided RL exposes a critical failure mode: algorithms capable of non-myopic reasoning collapse as proposition spaces scale, while scalable alternatives depend entirely on myopic shortcuts.

Decoupling candidate actions into reusable cacheable embeddings yields a 5x inference speedup, but collapses real-world rule adherence by more than 35 percentage points compared to joint cross-attention.

Long-horizon agents plateau under naive inference scaling unless compute is explicitly redirected from task workers to a dedicated meta-reasoning controller that manages execution flow.

General-purpose VLMs can achieve state-of-the-art zero-shot robot manipulation by directly interpreting observations and issuing actions, eliminating the need for specialized policies or grounding tools.

Video agents can now autonomously discover and fix their own reasoning flaws by re-watching videos and evolving their evidence-gathering strategies.

Robotic agents can now learn complex manipulation strategies from just a few demonstrations by treating them as evidence to be analyzed, not just training data.

Multi-agent LoRA systems can slash time-to-first-token by 3.1× without sacrificing role specialization by decoupling shared base KV caches from precomputed low-rank adapter updates.

LLM agent token consumption swings by over an order of magnitude on identical tasks, but dynamically tracking context inflation across execution segments slashes budget waste by 21.3% without making a single extra LLM call.

Extra test-time "thinking" cannot recover missing parametric facts, but teaching small reasoning models to recognize their own knowledge boundaries and selectively query larger models lets a 4B model outperform a 14B baseline at nearly a third of the serving cost.

Lazy-loading agent skills on demand doesn't just slash context window costs—it actively sharpens tool retrieval quality, all while incurring only a negligible hit to latency.

Real-time multi-stream interaction does not require bespoke model architectures: off-the-shelf LLMs can natively interleave perception and action in dynamic environments when configured with overlapping-memory inference coroutines.

Modifying executable agent harnesses rather than model weights allows test-time adaptation to generalize across unseen reasoning tasks without a single gradient step.

Shared workspace files can turn stateful LLM assistants into self-replicating malware vectors, silently compromising up to 80% of independent agents across eight consecutive hops via persistent memory.

Non-CUDA hardware no longer requires costly post-training pipelines: conditioning memory rewards strictly on adopted code enables zero-shot LLM agents to synthesize NPU kernels that outperform vendor libraries by up to 44x.

Just two incoming connections per agent completely break the finite-depth impossibility of single-parent aggregation, recovering the globally optimal predictor across arbitrary distributions in $O(d \log d)$ depth.

Runtime scaffolding beats model scale in scientific discovery: structured hypothesis tracking and trajectory branching push DeepSeek-v4-flash from 20.7% to 73.0% accuracy on blind symbolic regression, matching GPT-5.5 without relying on semantic domain clues.

Domain-specific agent evaluation no longer requires manual ground-truth curation: autonomous research and verification loops constrained by reusable expert rules yield grading rubrics that professional analysts prefer over expert baselines.

Parallel LLM scientific agents collaborate far more effectively when sharing raw evidence rather than hypotheses, unlocking a 9.5-point discovery quality gain and nearly doubling exploration speed without succumbing to premature consensus.

Frontier coding agents can now write 100% functionally correct GPU physics simulations, yet they match human-expert performance on just 22% of tasks—revealing a stark capability gap between generating working code and producing performant systems.

Long-horizon agent RL frequently stalls because fixed rewards cannot track shifting bottlenecks—co-evolving evaluation rubrics with targeted exploration skills directly breaks the repetitive failure loops of current policies.

Chasing large token-level teacher-student gaps in agent distillation is largely misguided: local divergence fails to predict task completion, whereas unpenalized minor deviations often trigger catastrophic multi-turn failure.

Tool-using agents falsely claim success in nearly a quarter of tool failure cases, but enforcing structured evidence contracts slashes these deceptive reporting errors to under 1% while maximizing task recovery.

Standard next-token prediction on an agent's self-generated explanations outperforms GRPO on SWE-bench in half the training updates—even bootstrapping on tasks where every initial rollout failed.

Standard pass rates hide severe technical debt: coding agents duplicate logic in over half of multi-turn task chains by turn 5 while progressively abandoning repository exploration.

Cracking LLM ideation requires more than high sampling temperatures: reinforcement learning over cognitive creativity axes expands scientific exploration diversity by 28% while boosting proposal originality by 66 percentage points.

Compiling mobile GUI navigation into deterministic commands turns slow, multi-turn VLM visual search into sub-second execution at zero token cost without sacrificing open-ended task completion.

Autonomous agents can drift into severe safety failures purely through benign, self-directed harness updates—no adversarial prompting required.

Across fifteen read vectors, standard containers and sandboxes completely fail to distinguish an autonomous coding agent from the compiler it executes, leaving proprietary dependencies exposed to exfiltration.

Multi-agent collaboration often performs worse than solo reasoning when peers hallucinate, but Bayesian tracking of advisor reliability rooted in internal belief states keeps collective performance strictly above autonomous baselines even under high misinformation.

Autonomous model post-training can outperform human-crafted instruct baselines on hard coding benchmarks while completely eliminating reward hacking once experimental exploration is governed by a sandboxed OS and a dynamic peer-review DAG.

Multi-agent collaboration bottlenecks when agent personas stay rigid, but dynamic, test-time strategy transplantation across system prompts enables heterogeneous LLMs to collectively align on the most effective reasoning path.

Human-engineered black-box optimizers have been eclipsed by a recursive dual-agent system that autonomously evolves both the target optimizer codebase and its own meta-design logic from scratch.

Autonomous LLM agents guided by early learning-curve forecasting can discover generalist EEG architectures that outperform state-of-the-art foundation models across 14 diverse clinical and affective benchmarks.

Parallel exploration beats sequential refinement when scaling agent test-time compute, but early-stage trajectory errors create persistent bottlenecks that deeper search and downstream model grafting struggle to overcome.

LLM agents can ditch unwieldy context windows entirely: compacting memory at every single turn matches or beats full-history baselines when guided by privileged full-context distillation.

Smarter agents fail harder when reading human-facing MCP error messages, blindly executing impossible developer commands to cause up to a 69-point collapse in task recovery.

Embodied agent bottlenecks in 3D environments stem from downstream physical execution rather than high-level reasoning once visual-symbolic grounding eliminates perceptual ambiguity.

Agent backtracking and parallel tree-search become practical across real-world tools when incremental local checkpoints are paired with automated remote compensating actions, driving a 15x task quality gain at just 3% overhead.

High benchmark accuracy masks catastrophic runtime drift: borrowing classic industrial reliability engineering—from FMEA to sequential monitoring—is essential to quantify when, where, and how autonomous agentic loops will fail.

Key contribution not extracted.

External agent scaffolding can be discarded after training by distilling evolved decision-making procedures directly into a model's native reasoning traces.

Over 70% of sparse autoencoder features exhibit fundamentally mismatched input and output semantics, breaking the common assumption that what activates a feature directly mirrors its downstream causal effect.

LLMs can scale to full 100,000-word novels without narrative collapse simply by offloading character arcs, plot history, and future constraints to an explicit, training-free state-tracking agent.

Language agents do not need to be verbose to be persuasive: decoupling strategic planning from expressive conciseness yields higher negotiation success while cutting dialogue token cost.

Chaining low-rank KV corrections directly across sequential agent workflows eliminates redundant prefills, slashing peak GPU memory by 3.7x and cutting time-to-first-token in half without sacrificing model accuracy.

Context compaction in agentic RL can run up to 5x faster simply by streaming the KV cache instead of flushing it—accidentally turning standard LLMs into recurrent agents that preserve evicted context purely through RL.

Value-driven optimistic exploration has historically crumbled under deep network approximations, but diagnosing and repairing failure modes in uncertainty propagation allows pure model-free RL to beat complex model-based exploration baselines.

AI scientific agents frequently solve empirical equations for entirely the wrong reasons, failing to identify the correct generative mechanism in over 64% of the cases where they successfully recover the observable phenomenal law.

Long-horizon in-context RL no longer requires exploding context windows—compressing past learning trajectories into a fixed-size latent buffer matches full Algorithm Distillation performance at a fraction of the memory cost.

Repeatedly pairing the same learning agents is enough to drive market prices toward monopoly levels, completely bypassing the need for observable retaliatory punishment.

Frozen 4B image generators can outperform proprietary models at multi-subject composition simply by letting an autonomous coding agent iteratively rewrite their Python orchestration harnesses.

Ditching autoregressive token generation in favor of continuous flow matching slashes tool-use latency by over 50× while beating proprietary MLLMs at image retouching.

A single agent can disentangle true mechanisms from computational twin errors by jointly learning them, unlocking more accurate mechanistic discovery in complex biological systems.

Internal interventions that boost tool-use accuracy often corrupt over half of baseline-correct decisions, revealing that behavioral improvement doesn't equate to functional repair.

DataMagic achieves over 95% success rate and an 83% quality improvement in data video generation by declaratively orchestrating multi-agent scene creation, surpassing even state-of-the-art LLMs.

Coding agents can now achieve up to 15% higher task completion rates by treating feedback as a persistent note that's followed until resolved, not just a one-off evaluation.

Personal GUI agents waste most of their retrieved history; this framework activates experience only when it's actually applicable, boosting performance and efficiency.

Agents can achieve up to 5.8% higher success rates by learning a reusable "experience tree" directly from data, bypassing LLM-generated skills and enabling true generalization.

Iterative intent denoising enables decentralized multi-agent pathfinding agents to coordinate complex joint actions, overcoming the limitations of one-shot sampling and achieving near-perfect task completion.

AI systems can auto-evolve by learning from their own failures, achieving 20.5% better performance with 65.5% fewer rollouts than existing methods.

Enterprise AI agents can now generate valid, high-fidelity tabular data for training and evaluation without needing access to sensitive schemas or real systems, bypassing critical industry bottlenecks.