Tool Use & Agents - Weekly Roundup

A simple visual harness transforms a general multimodal model into a perfect reasoner on ARC-AGI-3, completing all games with drastically fewer actions than humans.

Achieve state-of-the-art LLM-guided program evolution for under $2, drastically cutting costs for complex optimization tasks.

LLMs can recursively rewrite their own solutions from specialized environments into generalizable training data, boosting performance on complex tasks by over 34% and improving pass@3 by up to 17 percentage points.

LLM agents will pick a worse item from a "preferred" source over a better item from a "dispreferred" source two-thirds of the time, revealing a critical bias that can be reduced by providing more information.

A single set of learned manipulation skills transfers to the real world with 78% success, bypassing the need for task-specific fine-tuning or re-training on real data.

Achieve state-of-the-art LLM-guided program evolution for under $2, drastically cutting costs for complex optimization tasks.

LLMs can recursively rewrite their own solutions from specialized environments into generalizable training data, boosting performance on complex tasks by over 34% and improving pass@3 by up to 17 percentage points.

LLM agents will pick a worse item from a "preferred" source over a better item from a "dispreferred" source two-thirds of the time, revealing a critical bias that can be reduced by providing more information.

A single set of learned manipulation skills transfers to the real world with 78% success, bypassing the need for task-specific fine-tuning or re-training on real data.

Multimodal agents can achieve >90% reasoning latency reduction and comparable task performance by selecting from pre-defined reasoning steps instead of generating them freely.

LLM-driven heuristic design now learns from noisy system logs to uncover hidden system dynamics, outperforming prior methods in real-world resource assignment.

Robot skills can be dramatically improved post-training by autonomously generating practice tasks, diagnosing failures with privileged simulator states, and iteratively refining symbolic skills and system prompts, achieving near-perfect success on novel manipulation tasks.

Persistent world models can now track objects even when they leave the agent's view, thanks to a novel actor-observer decoupling mechanism.

Robot agents achieve 14% higher success rates and use 65% fewer tokens by composing primitives and performing local retries within an interactive code execution framework.

Even the largest open-weight LLMs fail to generate accurate cybersecurity CLI commands over 58% of the time, but fine-tuning with verifiable rewards on KaliBench closes the gap with massive MoE models.

LLM workflows can be optimized without human labels by treating agent assignment and task decomposition as a bidding process, slashing optimization costs.

Enterprise data agents struggle to perform complex reasoning and consequential actions across massive, multi-table datasets, with even top models failing over 65% of realistic tasks.

Post-training LLMs for agentic tasks can drastically reduce solution coverage, a "Sharpening Tax" that a novel Bayesian sampler can mitigate.

Robust optimal policies and values for RMDPs are now computable in strongly polynomial time via a single LP, unifying and extending prior results across diverse uncertainty models.

LLMs can be tricked into leaking sensitive data to attackers simply by being asked to fetch information from seemingly legitimate websites.

Mutation-enhanced skill evolution unlocks significant improvements in Lean theorem proving by intelligently sampling mathematical concepts when proof generation stalls.

Verifier-guided RL unlocks a 34% accuracy boost in mathematical reasoning by directly penalizing incorrect intermediate steps, not just final answers.

LLM agents can now build lasting, reusable "competence" for specific knowledge sources, moving beyond simple retrieval to deep, adaptive understanding.

A robot can learn its partner's hidden physical limitations simply by watching it collaborate, enabling seamless coordination on novel tasks without explicit communication.

A single 10M-parameter Transformer policy achieves near-perfect win rates in competitive Super Smash Bros. Melee, outperforming specialized AIs and even a zero-delay opponent.

A single-epoch GRPO training regime on a novel agentic forecasting environment transforms LLMs into evidence-disciplined forecasters that outperform frontier models on complex real-world predictions at 5% of inference cost.

Diffusion models can now achieve one-step, efficient action generation for online multi-agent coordination, shattering previous scalability barriers.

A simple visual harness transforms a general multimodal model into a perfect reasoner on ARC-AGI-3, completing all games with drastically fewer actions than humans.

Humanoid robots struggle to bridge the gap between selecting the right tool and executing complex tasks, with current policies showing significant performance degradation on unseen tools and unrelated instructions.

A curriculum-trained 1B parameter model fails to generalize tool use, despite scoring identically to a smaller, simpler model on lenient benchmarks, due to a missing prior erased by its web-heavy training.

Coding agents can achieve over 4x throughput gains with a novel retrieval-based speculative decoding framework that intelligently sources and formats reusable text and adapts draft lengths.

LLM agents can be turned into powerful verifiers by giving them a workspace and evidence tools, outperforming baselines on complex long-horizon tasks by resolving disagreements and challenging consensus.

Rejection-driven resubmission, not just population growth, is a primary driver of overwhelming reviewer burden in academic research.

Ultra-long video grounding is solved by evolving reusable agent skills and tools from VLM reasoning, bypassing parameter updates and separate planning models.

A single capable verifier can dramatically improve terminal agent action reliability, achieving higher success rates at lower costs than simply generating more trajectories.

Learned world models can achieve real-time control in live games, outperforming behavioral cloning and maintaining high-frequency action loops.

Most failed multi-turn agent trajectories contain a single, recoverable pivotal mistake that, if addressed, can restore task success, a phenomenon PivotOPD exploits for significant performance gains.

State-of-the-art VLMs struggle to execute scientific research workflows, revealing a critical gap in their ability to interact with specialized software and produce verifiable results.

LLMs can overcome knowledge gaps in evolutionary search by dynamically co-evolving solutions and targeted web queries, leading to substantial gains in scientific discovery.

Robots can now learn from physical failures in real-time, autonomously revising their own capabilities to achieve unprecedented long-horizon manipulation success.

Faster computer-use agents are bottlenecked by their speed, but existing benchmarks are irreproducible; this new standardized framework reveals counterintuitive speed-efficiency trade-offs and that current open-weight models lag behind.

LLM agents can improve their decision-making by learning from their own diagnosed failures, with first-proposal corrections boosting success rates by over 30 percentage points.

A listwise agent inspecting top candidates holistically boosts patch retrieval accuracy by over 27% compared to treating them independently.

Standard information retrieval methods can match LLM-mediated skill selection for agents, slashing per-task costs from $51 to $27.

Visual perception agents can achieve significant performance gains by actively seeking information beyond a single glance, outperforming static models by up to 30 points on challenging benchmarks.

Imagined failures in a world model can pinpoint exactly which robot skill components need new demonstrations, boosting success from 13% to 75% with minimal data.

LLM agents trained solely in rule-generated fictional worlds achieve state-of-the-art transfer to real-world search tasks, outperforming agents trained on actual data.

A single graph edge misspecification can transform a seemingly robust causal action verifier into a dangerous liability, executing harmful actions with internally valid but misleading certificates.

AutoResearch systems can achieve up to 28% fewer errors by making competing behaviors visible and actively steering edits to resolve their inherent conflicts.

Lightweight sentinels can learn to proactively intervene in coding agent actions, boosting task success by up to 14% by preventing costly error propagation.

Parallel search agents can now achieve higher accuracy than generative aggregators by leveraging TRACE to intelligently select the best trajectory from aggregated cross-rollout evidence.

Autonomous agents can recursively self-improve LLM-generated games, achieving superhuman performance and drastically cutting generation costs by internalizing development experience.

Frontier LLMs can be trained to selectively ignore unreliable external guidance, a capability crucial for robust agentic behavior beyond simple task accuracy.

Agentic discovery yields a deletion-only DNA designer that outperforms greedy and random approaches, achieving substantial sequence reduction while preserving predicted function.

Most multimodal models fail to truly understand tool use in real-world egocentric videos, with even Gemini-3.1-Pro struggling to ground actions in visual evidence.

LLM agents can now adapt their action spaces dynamically from natural language, bypassing the need for pre-defined finite fields and accelerating inference through compositional policies.

Clinical LLM agents can now autonomously gather evidence from longitudinal medical records and imaging, achieving superior accuracy to GPT-5.4 and Claude Opus.

LLM agents abandon their own declared plans up to 78% of the time, but enforcing faithful execution via pattern-specific scaffolding nearly doubles benchmark success rates.

Teaching agents *what* unstated information to hunt down—rather than just *when* to act—enables small models to beat 15× larger baselines at multi-hop task completion while asking fewer questions.

LLM agents waste immense token budgets repeatedly rediscovering identical cross-document links on every query—pre-indexing corpora into an offline entity-document graph solves fragmented multi-hop retrieval while actually lowering token costs.

Language models that edit their own context window like a mutable file beat external agent scaffolding across long-horizon tasks while cutting compute by up to 59%.

Heterogeneous visual agents can boost downstream performance by up to 4.9 points across diverse tasks simply by reading from a shared, self-organizing 2B memory tree—without updating a single host model weight.

Computer-use agents can bypass brittle, hand-engineered APIs entirely by learning when to substitute mouse clicks with native shell commands, yielding a 14.8-point leap on OSWorld.

Robots no longer need human intervention to learn from their mistakes: foundation-model-guided exploration over modular skills can autonomously synthesize corrective rollouts that distill directly into low-level policy weights.

Translating learned strategic principles into an external runtime harness yields a 12-point performance boost over directly feeding those same principles into an agent's context window.

Frozen tabular foundation models can dominate competitive ML benchmarks without parameter updates if an LLM agent is tasked strictly with evolving the semantic data pipeline around them.

Today's frontier agents virtually collapse in professional engineering environments, solving just 3.6% of multi-software workflows when held to real-world physical and geometric constraints.

Training a small advisor to steer frozen frontier LLMs actually degrades if it learns from every post-hoc correction—selectively distilling only the advice that causally shifts execution yields up to a 6.4% gain and robust cross-family model transfer.

Append-only memory guarantees attention collapse in long-horizon video reasoning, but actively rewriting a bounded working memory from an external filesystem boosts hard-subset task accuracy by over 20%.

Self-improving coding agents do not require downstream reward signals to evolve: bootstrapping purely from historical self-modification traces matches the performance of benchmark-guided search at a fraction of the cost.

A 220× acceleration of LTL-guided RL exposes a critical failure mode: algorithms capable of non-myopic reasoning collapse as proposition spaces scale, while scalable alternatives depend entirely on myopic shortcuts.

Decoupling candidate actions into reusable cacheable embeddings yields a 5x inference speedup, but collapses real-world rule adherence by more than 35 percentage points compared to joint cross-attention.

Long-horizon agents plateau under naive inference scaling unless compute is explicitly redirected from task workers to a dedicated meta-reasoning controller that manages execution flow.

General-purpose VLMs can achieve state-of-the-art zero-shot robot manipulation by directly interpreting observations and issuing actions, eliminating the need for specialized policies or grounding tools.

Video agents can now autonomously discover and fix their own reasoning flaws by re-watching videos and evolving their evidence-gathering strategies.

Robotic agents can now learn complex manipulation strategies from just a few demonstrations by treating them as evidence to be analyzed, not just training data.

Multi-agent LoRA systems can slash time-to-first-token by 3.1× without sacrificing role specialization by decoupling shared base KV caches from precomputed low-rank adapter updates.

LLM agent token consumption swings by over an order of magnitude on identical tasks, but dynamically tracking context inflation across execution segments slashes budget waste by 21.3% without making a single extra LLM call.

Extra test-time "thinking" cannot recover missing parametric facts, but teaching small reasoning models to recognize their own knowledge boundaries and selectively query larger models lets a 4B model outperform a 14B baseline at nearly a third of the serving cost.

Lazy-loading agent skills on demand doesn't just slash context window costs—it actively sharpens tool retrieval quality, all while incurring only a negligible hit to latency.

Real-time multi-stream interaction does not require bespoke model architectures: off-the-shelf LLMs can natively interleave perception and action in dynamic environments when configured with overlapping-memory inference coroutines.

Modifying executable agent harnesses rather than model weights allows test-time adaptation to generalize across unseen reasoning tasks without a single gradient step.

Shared workspace files can turn stateful LLM assistants into self-replicating malware vectors, silently compromising up to 80% of independent agents across eight consecutive hops via persistent memory.

Non-CUDA hardware no longer requires costly post-training pipelines: conditioning memory rewards strictly on adopted code enables zero-shot LLM agents to synthesize NPU kernels that outperform vendor libraries by up to 44x.

Just two incoming connections per agent completely break the finite-depth impossibility of single-parent aggregation, recovering the globally optimal predictor across arbitrary distributions in $O(d \log d)$ depth.

Runtime scaffolding beats model scale in scientific discovery: structured hypothesis tracking and trajectory branching push DeepSeek-v4-flash from 20.7% to 73.0% accuracy on blind symbolic regression, matching GPT-5.5 without relying on semantic domain clues.

Domain-specific agent evaluation no longer requires manual ground-truth curation: autonomous research and verification loops constrained by reusable expert rules yield grading rubrics that professional analysts prefer over expert baselines.

Parallel LLM scientific agents collaborate far more effectively when sharing raw evidence rather than hypotheses, unlocking a 9.5-point discovery quality gain and nearly doubling exploration speed without succumbing to premature consensus.

Frontier coding agents can now write 100% functionally correct GPU physics simulations, yet they match human-expert performance on just 22% of tasks—revealing a stark capability gap between generating working code and producing performant systems.

Long-horizon agent RL frequently stalls because fixed rewards cannot track shifting bottlenecks—co-evolving evaluation rubrics with targeted exploration skills directly breaks the repetitive failure loops of current policies.

Chasing large token-level teacher-student gaps in agent distillation is largely misguided: local divergence fails to predict task completion, whereas unpenalized minor deviations often trigger catastrophic multi-turn failure.

Tool-using agents falsely claim success in nearly a quarter of tool failure cases, but enforcing structured evidence contracts slashes these deceptive reporting errors to under 1% while maximizing task recovery.

Standard next-token prediction on an agent's self-generated explanations outperforms GRPO on SWE-bench in half the training updates—even bootstrapping on tasks where every initial rollout failed.

Standard pass rates hide severe technical debt: coding agents duplicate logic in over half of multi-turn task chains by turn 5 while progressively abandoning repository exploration.

Cracking LLM ideation requires more than high sampling temperatures: reinforcement learning over cognitive creativity axes expands scientific exploration diversity by 28% while boosting proposal originality by 66 percentage points.

Compiling mobile GUI navigation into deterministic commands turns slow, multi-turn VLM visual search into sub-second execution at zero token cost without sacrificing open-ended task completion.

Autonomous agents can drift into severe safety failures purely through benign, self-directed harness updates—no adversarial prompting required.

Across fifteen read vectors, standard containers and sandboxes completely fail to distinguish an autonomous coding agent from the compiler it executes, leaving proprietary dependencies exposed to exfiltration.

Multi-agent collaboration often performs worse than solo reasoning when peers hallucinate, but Bayesian tracking of advisor reliability rooted in internal belief states keeps collective performance strictly above autonomous baselines even under high misinformation.

Autonomous model post-training can outperform human-crafted instruct baselines on hard coding benchmarks while completely eliminating reward hacking once experimental exploration is governed by a sandboxed OS and a dynamic peer-review DAG.

Multi-agent collaboration bottlenecks when agent personas stay rigid, but dynamic, test-time strategy transplantation across system prompts enables heterogeneous LLMs to collectively align on the most effective reasoning path.

Human-engineered black-box optimizers have been eclipsed by a recursive dual-agent system that autonomously evolves both the target optimizer codebase and its own meta-design logic from scratch.

Autonomous LLM agents guided by early learning-curve forecasting can discover generalist EEG architectures that outperform state-of-the-art foundation models across 14 diverse clinical and affective benchmarks.

Parallel exploration beats sequential refinement when scaling agent test-time compute, but early-stage trajectory errors create persistent bottlenecks that deeper search and downstream model grafting struggle to overcome.

LLM agents can ditch unwieldy context windows entirely: compacting memory at every single turn matches or beats full-history baselines when guided by privileged full-context distillation.