Tool Use & Agents - Weekly Roundup

A simple visual harness transforms a general multimodal model into a perfect reasoner on ARC-AGI-3, completing all games with drastically fewer actions than humans.

Amortized agentic policy discovery enables personalized test-time scaling to satisfy complex, multidimensional user requirements, outperforming single-objective optimization on unseen profiles.

A novel contrastive feedback mechanism allows LLM agents to learn skill proposals that are aligned with the current actor's behavior without requiring expensive skill evaluations.

LLM agents can dynamically evolve their skill libraries, preventing obsolescence and improving long-horizon task performance by up to 7.8% through a fitness-driven lifecycle.

LLM agents can now reliably track and fulfill complex deliverable requirements across numerous files without losing context, thanks to an environment-side "tab" that acts as a persistent, shared task ledger.

Amortized agentic policy discovery enables personalized test-time scaling to satisfy complex, multidimensional user requirements, outperforming single-objective optimization on unseen profiles.

A novel contrastive feedback mechanism allows LLM agents to learn skill proposals that are aligned with the current actor's behavior without requiring expensive skill evaluations.

LLM agents can dynamically evolve their skill libraries, preventing obsolescence and improving long-horizon task performance by up to 7.8% through a fitness-driven lifecycle.

LLM agents can now reliably track and fulfill complex deliverable requirements across numerous files without losing context, thanks to an environment-side "tab" that acts as a persistent, shared task ledger.

AI agents are now building climate models that outperform published research and offer novel hypotheses for long-standing scientific debates.

An LLM-based agent achieves 18.3% out-of-sample returns in stock options trading by learning to select structured strategies, but surprisingly, retaining news during RL training halves performance.

Frontier LLMs can achieve substantial recursive self-improvement, closing nearly half of the performance gap on complex benchmarks by jointly optimizing data, training, and execution harnesses within a unified, agent-native environment.

Uncoded crash factors like phone use and medical episodes are vastly undercounted in police records, with narratives revealing 2x more injury crashes than coded fields.

A frozen graph backbone effectively preserves past structural knowledge in continual multi-agent reinforcement learning, dramatically reducing forgetting and boosting transfer across tasks with distinct graph structures.

Planning in visual world models can be accelerated by $5\times$ by pruning $50\%$ of spatial tokens based on their influence on the planning cost gradient, without sacrificing performance on most control tasks.

AI is poised to revolutionize ARPES, moving beyond data analysis to enable fully autonomous, self-driving experimental labs.

Intermediate retrieval signals, not just final outcomes, unlock efficient credit assignment for training LLM search agents.

AI agents can now build and reuse persistent, artifact-specific knowledge, dramatically improving their ability to find relevant information for complex tasks and reducing operational costs.

LLMs can learn to actively compute evidence by routing through retrieval and computation, dramatically improving performance on complex, multi-source reasoning tasks.

Physical robot learning can now autonomously discover and exploit informative experiments to continuously improve foundation models, achieving 77% success on RoboCasa365 and zero-shot transfer to real hardware.

Frontier multimodal agents struggle to autonomously acquire and utilize task-relevant information in unfamiliar physical environments, failing 77% of the time on a new exploration benchmark.

Key contribution not extracted.

Real-world coding agent usage reveals that task flow, not just task completion, dramatically impacts agent cost and effectiveness, suggesting current benchmarks are misaligned with user behavior.

Trillion-parameter agentic RL refits are now 12-40x faster, shrinking an 87.5-minute full checkpoint transfer to 150 seconds by sending only bit-exact weight deltas.

A single, lightweight model-internal mechanism can effectively bridge the gap between corpus retrieval and long-context inference, outperforming specialized SOTA systems on both.

Storing knowledge directly in LLM parameters offers a flexible, efficient alternative to in-context learning, bypassing context length limitations and repeated encoding costs.

Agents can learn to act reliably in new environments by generating their own practice tasks and distilling reusable heuristics from self-identified failures, achieving significant performance gains without human supervision or pre-defined benchmarks.

Personal LLM agents can significantly improve recommendation quality by mediating platform rankings with cross-platform history, but naive mediation introduces substantial harmful overrides.

Embodied agents can now reliably achieve complex, multi-stage goals in dynamic environments by learning to predict and ground targets from decoupled visual context, even when the goal is out of view.

Latency in voice assistants plummets by 20% by speculatively executing tool calls *during* speech recognition, not after.

Sequential recommenders can dramatically boost catalog diversity by adaptively pruning action sets, achieving up to 5.21x diversity gains without sacrificing session depth.

Context compaction in long-horizon agents is surprisingly insensitive to recent agent behavior, with simple "has-written" heuristics measuring trajectory phase rather than predictive power.

Parallel LLM agents can finally achieve near-linear speedups by strategically parallelizing only when token-based cost estimates predict a net gain, eliminating redundant work and reconciliation overhead.

LLM agents can autonomously optimize RAG pipelines by diagnosing retrieval vs. generation failures, outperforming traditional hyperparameter tuning and achieving Pareto efficiency.

LLM agents achieve substantial performance gains by jointly learning credit at both the turn and token generation levels, overcoming limitations of existing single-level credit assignment methods.

LLM-generated heuristics can now guarantee optimal plans, outperforming specialized planners by combining generative power with formal verification.

LLMs can decode partner intents but struggle to translate this knowledge into cooperative actions, with explicit action recommendations proving far more effective than abstract rule statements.

LLM agents struggle to translate general capabilities into cost-efficient, reusable solutions, often failing to outperform simple distillation baselines despite significant zero-shot prowess.

Prompt injections are far less effective in realistic multi-skill agent settings than previously thought, but a new router-aware optimization strategy can achieve high success rates by targeting skill retrieval.

Tool-using agents frequently act without sufficient evidence, even when capable of strong static assessment, due to premature execution and incomplete investigation.

Embodied AI agents can now continuously self-improve by co-evolving their policies, training curricula, and judges, overcoming the bottleneck of static evaluation.

Web agents trained with adaptive adversarial simulation achieve 33.6% higher completion rates against unseen prompt injection attacks, even outperforming frontier adversaries.

Discovering molecules with rare properties can now be an order of magnitude faster by testing complex mixtures and deconvoluting activity, slashing experimental costs from $\mathcal{O}(d)$ to $\mathcal{O}(\log d)$ or $\mathcal{O}(1)$.

LLM agents achieve dramatic efficiency gains in long-horizon research by offloading conversational history to a central harness, enabling focused, role-specific contexts for each task.

AI agents in a simulated scientific ecosystem rediscovered over 60% of key findings from recent research papers, suggesting a viable path to autonomous open-ended scientific discovery.

LLMs can now autonomously debug complex chip designs by translating natural language queries into SQL against massive simulation dumps.

Semantic decision engines routinely fail network deadlines and select infeasible actions, with most claims of real-time performance unsupported by evidence.

LLM agents can now dynamically curate multimodal memory on-demand, achieving optimal performance-cost-latency trade-offs by intelligently choosing between pre-computed and runtime-generated evidence.

LLM agents in decentralized marketplaces are demonstrably exploitable, with adversarial prompting enabling over 50% fraudulent transaction completion and doubling agent profits.

Learned corner selection for cooperative guards dramatically boosts deployment efficiency and agent utilization in partially observable environments, outperforming established methods without sacrificing formal guarantees.

Local AI models now predict user well-being from passive computer activity, offering explainable feedback directly within a workplace sensing application.

Network traffic alone can reveal LLM agent risks and granular behaviors, but current analysis methods fail to distinguish subtle malicious activities or semantically similar scenarios.

Hierarchical procedural understanding unlocks a 9.6%p performance boost in proactive AI assistance, enabling adaptive guidance that outperforms in-context learning by 57.1%p.

Agentic loops can now reliably stop with provable guarantees on proposal quality, even with imperfect verifiers.

Prompt injection defenses can degrade LLM agent utility by altering benign behavior; RAISED achieves robustness without this cost through self-distillation on generated scenarios.

Flowcharts break silently across aspect ratios, but a new agentic pipeline with VLM critics now guarantees connectivity and editability.

Robots can now learn complex manipulation tasks from a single demonstration video by recursively accessing fine-grained sub-event details, boosting success rates by up to 9%.

Web agents can now achieve state-of-the-art performance and robust test-time scaling by learning to self-verify their actions, bypassing the need for costly external judges during deployment.

LLMs forget user intent after a rejected suggestion, treating it as active even when superseded, leading to task failure.

Achieve state-of-the-art LLM-guided program evolution for under $2, drastically cutting costs for complex optimization tasks.

LLMs can recursively rewrite their own solutions from specialized environments into generalizable training data, boosting performance on complex tasks by over 34% and improving pass@3 by up to 17 percentage points.

LLM agents will pick a worse item from a "preferred" source over a better item from a "dispreferred" source two-thirds of the time, revealing a critical bias that can be reduced by providing more information.

A single set of learned manipulation skills transfers to the real world with 78% success, bypassing the need for task-specific fine-tuning or re-training on real data.

Multimodal agents can achieve >90% reasoning latency reduction and comparable task performance by selecting from pre-defined reasoning steps instead of generating them freely.

LLM-driven heuristic design now learns from noisy system logs to uncover hidden system dynamics, outperforming prior methods in real-world resource assignment.

Robot skills can be dramatically improved post-training by autonomously generating practice tasks, diagnosing failures with privileged simulator states, and iteratively refining symbolic skills and system prompts, achieving near-perfect success on novel manipulation tasks.

Persistent world models can now track objects even when they leave the agent's view, thanks to a novel actor-observer decoupling mechanism.

Robot agents achieve 14% higher success rates and use 65% fewer tokens by composing primitives and performing local retries within an interactive code execution framework.

Even the largest open-weight LLMs fail to generate accurate cybersecurity CLI commands over 58% of the time, but fine-tuning with verifiable rewards on KaliBench closes the gap with massive MoE models.

LLM workflows can be optimized without human labels by treating agent assignment and task decomposition as a bidding process, slashing optimization costs.

Enterprise data agents struggle to perform complex reasoning and consequential actions across massive, multi-table datasets, with even top models failing over 65% of realistic tasks.

Post-training LLMs for agentic tasks can drastically reduce solution coverage, a "Sharpening Tax" that a novel Bayesian sampler can mitigate.

Robust optimal policies and values for RMDPs are now computable in strongly polynomial time via a single LP, unifying and extending prior results across diverse uncertainty models.

LLMs can be tricked into leaking sensitive data to attackers simply by being asked to fetch information from seemingly legitimate websites.

Mutation-enhanced skill evolution unlocks significant improvements in Lean theorem proving by intelligently sampling mathematical concepts when proof generation stalls.

Verifier-guided RL unlocks a 34% accuracy boost in mathematical reasoning by directly penalizing incorrect intermediate steps, not just final answers.

LLM agents can now build lasting, reusable "competence" for specific knowledge sources, moving beyond simple retrieval to deep, adaptive understanding.

A robot can learn its partner's hidden physical limitations simply by watching it collaborate, enabling seamless coordination on novel tasks without explicit communication.

A single 10M-parameter Transformer policy achieves near-perfect win rates in competitive Super Smash Bros. Melee, outperforming specialized AIs and even a zero-delay opponent.

A single-epoch GRPO training regime on a novel agentic forecasting environment transforms LLMs into evidence-disciplined forecasters that outperform frontier models on complex real-world predictions at 5% of inference cost.

Diffusion models can now achieve one-step, efficient action generation for online multi-agent coordination, shattering previous scalability barriers.

A simple visual harness transforms a general multimodal model into a perfect reasoner on ARC-AGI-3, completing all games with drastically fewer actions than humans.

Humanoid robots struggle to bridge the gap between selecting the right tool and executing complex tasks, with current policies showing significant performance degradation on unseen tools and unrelated instructions.

A curriculum-trained 1B parameter model fails to generalize tool use, despite scoring identically to a smaller, simpler model on lenient benchmarks, due to a missing prior erased by its web-heavy training.

Coding agents can achieve over 4x throughput gains with a novel retrieval-based speculative decoding framework that intelligently sources and formats reusable text and adapts draft lengths.

LLM agents can be turned into powerful verifiers by giving them a workspace and evidence tools, outperforming baselines on complex long-horizon tasks by resolving disagreements and challenging consensus.

Rejection-driven resubmission, not just population growth, is a primary driver of overwhelming reviewer burden in academic research.

Ultra-long video grounding is solved by evolving reusable agent skills and tools from VLM reasoning, bypassing parameter updates and separate planning models.

A single capable verifier can dramatically improve terminal agent action reliability, achieving higher success rates at lower costs than simply generating more trajectories.

Learned world models can achieve real-time control in live games, outperforming behavioral cloning and maintaining high-frequency action loops.

Most failed multi-turn agent trajectories contain a single, recoverable pivotal mistake that, if addressed, can restore task success, a phenomenon PivotOPD exploits for significant performance gains.

State-of-the-art VLMs struggle to execute scientific research workflows, revealing a critical gap in their ability to interact with specialized software and produce verifiable results.

LLMs can overcome knowledge gaps in evolutionary search by dynamically co-evolving solutions and targeted web queries, leading to substantial gains in scientific discovery.

Robots can now learn from physical failures in real-time, autonomously revising their own capabilities to achieve unprecedented long-horizon manipulation success.

Faster computer-use agents are bottlenecked by their speed, but existing benchmarks are irreproducible; this new standardized framework reveals counterintuitive speed-efficiency trade-offs and that current open-weight models lag behind.

LLM agents can improve their decision-making by learning from their own diagnosed failures, with first-proposal corrections boosting success rates by over 30 percentage points.

A listwise agent inspecting top candidates holistically boosts patch retrieval accuracy by over 27% compared to treating them independently.

Standard information retrieval methods can match LLM-mediated skill selection for agents, slashing per-task costs from $51 to $27.

Visual perception agents can achieve significant performance gains by actively seeking information beyond a single glance, outperforming static models by up to 30 points on challenging benchmarks.

Imagined failures in a world model can pinpoint exactly which robot skill components need new demonstrations, boosting success from 13% to 75% with minimal data.

LLM agents trained solely in rule-generated fictional worlds achieve state-of-the-art transfer to real-world search tasks, outperforming agents trained on actual data.

A single graph edge misspecification can transform a seemingly robust causal action verifier into a dangerous liability, executing harmful actions with internally valid but misleading certificates.

AutoResearch systems can achieve up to 28% fewer errors by making competing behaviors visible and actively steering edits to resolve their inherent conflicts.

Lightweight sentinels can learn to proactively intervene in coding agent actions, boosting task success by up to 14% by preventing costly error propagation.

Parallel search agents can now achieve higher accuracy than generative aggregators by leveraging TRACE to intelligently select the best trajectory from aggregated cross-rollout evidence.