Tool Use & Agents - Weekly Roundup

Proactive agents can now be rigorously evaluated in real-world scenarios, revealing critical insights into their performance drivers.

Reward functions that generalize across environments can be learned more effectively by strategically leveraging heterogeneous feedback modalities, leading to substantial improvements in agent performance.

Watermarking LLM-agent trajectories just got a major upgrade—TRACE achieves near-perfect detection without sacrificing performance, even under aggressive adversarial conditions.

Harnesses can evolve in real-time during evaluation, leading to significant performance gains without retraining the underlying model.

Workflows can now be seen as dynamic knowledge objects, not just static processes, revolutionizing how we manage LLM interactions.

Proactive agents can now be rigorously evaluated in real-world scenarios, revealing critical insights into their performance drivers.

Reward functions that generalize across environments can be learned more effectively by strategically leveraging heterogeneous feedback modalities, leading to substantial improvements in agent performance.

Watermarking LLM-agent trajectories just got a major upgrade—TRACE achieves near-perfect detection without sacrificing performance, even under aggressive adversarial conditions.

Harnesses can evolve in real-time during evaluation, leading to significant performance gains without retraining the underlying model.

Workflows can now be seen as dynamic knowledge objects, not just static processes, revolutionizing how we manage LLM interactions.

Memory compaction in LLMs is fundamentally flawed, with critical information often discarded before it's needed, revealing a systemic inefficiency across all layers.

WebSwarm's innovative recursive delegation allows agents to not only search but also adaptively collaborate, leading to superior performance in complex web search tasks.

Trustworthy agentic AI evaluation in decentralized energy markets hinges on balancing market utility and safety, revealing critical vulnerabilities in reward-maximizing strategies.

Users who depend on manual context attachment experience a dramatic drop in task success, revealing a critical divide in AI utility that could reshape how we design intelligent systems.

Balancing session-centric scheduling can boost LLM cluster throughput by up to 16% without sacrificing token reuse.

Agents can now work independently on data changes while humans maintain oversight, revolutionizing collaborative data management.

Shifting the error landscape in compliance management, this pipeline reveals that a single misidentified asset can lead to irrelevant vulnerabilities, making risk assessment more visible and manageable.

Invisible perturbations can lead to a staggering 75.8% information loss in agentic crawlers without altering the human-visible content.

Compact and informative schemas generated by ASMR can revolutionize the way ship maintenance reports are authored, leading to more actionable insights.

Prismata cuts attack success rates dramatically while ensuring web agents can still perform their intended tasks without developer input.

TokenWall slashes the attack success rate to 12.5% while ensuring a 97.4% pass rate for benign interactions, all with just 0.69 seconds of added latency.

Analysts trust AI predictions but still rely heavily on traditional methods, highlighting a critical gap in AI integration for crime linkage analysis.

Self-evolving LLM agents can slash latency by up to 62% while significantly boosting reliability in industrial applications.

A proactive memory agent can significantly enhance decision-making in long-horizon tasks by preventing critical information from being forgotten.

CausalDS reveals that LLMs can navigate complex causal reasoning tasks while effectively managing uncertainty and abstention, a critical skill for real-world data analysis.

Bypassing database drivers can lead to up to 27x speedups in analytical workloads by directly reading storage files with LLM-assisted code synthesis.

LLM-generated skills fail to outperform basic task prompts in data science workflows, challenging the assumption that automated skill generation enhances AI performance.

Penalizing the decision-making path while rewarding the outcome can drastically reduce operational violations in real-world agent interactions.

Transforming data systems from passive repositories into active agents could redefine the landscape of autonomous automation and its safety protocols.

Easier tasks can sabotage the learning of harder tasks in multi-task RL, but a new entropy-aware optimization strategy can turn this challenge into an advantage.

With over 216,000 skills sourced from both academic and community contributions, SkillCenter transforms the landscape of operational knowledge for autonomous AI agents.

Structural designers thrive on friction, and interactive AI can enhance creativity by preserving the reflective challenges of the design process.

Over 40% improvement in analytical efficiency could revolutionize how researchers conduct trajectory inference in single-cell transcriptomics.

The orchestration layer can slash AI task costs by over 40% without sacrificing quality, fundamentally reshaping how enterprises approach agentic AI deployment.

Taint-style vulnerabilities in MCP servers are not only common but also require innovative mitigation strategies like SPELLSMITH, which outperforms conventional fixes.

A new seven-level harm scale reveals hidden vulnerabilities in AI agent defenses that binary metrics overlook, exposing risks even when attack-success rates appear low.

STRACE transforms noisy execution traces into precise optimization signals, leading to a 42.5% to 58.5% success rate improvement in agent performance.

Concentrating model capacity on delegation roles can yield substantial performance gains in hierarchical search agents, revealing a critical bottleneck in task decomposition.

Tool over-calling can be reduced by nearly 4% through a novel calibration method that reveals the hidden dynamics of token-level influences in multi-teacher training.

Self-evolving LLM agents can drastically reduce reasoning overhead by transforming atomic actions into reusable Standard Operating Procedures, leading to higher success rates and fewer interaction rounds.

Attackers can exploit LLMs' tendency to hallucinate resource identifiers, enabling scalable untargeted promptware attacks that could establish a botnet.

The unique challenges posed by agentic AI demand a fresh governance approach, as traditional frameworks may no longer suffice.

Silent policy violations in tool-using LLMs can be mitigated by deterministic gates, improving success rates by over 12 percentage points in critical tasks.

Transforming AI agent exploration into a deterministic workflow can slash operational costs by over 70% while doubling incident handling capacity.

Achieving a staggering 96.5% human acceptance rate, EmbodiedGen V2 transforms how we create and utilize 3D environments for embodied AI training.

Agents can significantly expand their capabilities through cooperative affordances, transforming how we design multi-agent systems in robotics.

FRAMe achieves up to 99% validity in easy scenarios, showcasing how LLMs can seamlessly align autonomous flight planning with human preferences.

Optimization performance varies significantly by workload, challenging the notion that larger models are always superior in coding tasks.

AI's role in code review is not a simple enhancement; it hinges on human expertise and the review process structure, revealing a complex interplay that challenges prevailing assumptions.

Static safety policies fail in offensive security, with ScopeJudge revealing that context-aware monitoring is crucial to avoid costly violations.

AgentLocate reveals not just which agent failed, but also the critical moment when the system first went off track, outperforming traditional methods in efficiency and accuracy.

Verifiable environments can empower web agents to self-evolve, achieving competitive performance without the need for external teacher models.

Jet-Long achieves a remarkable balance between short-context fidelity and long-context performance, outperforming leading models while remaining hyperparameter-resilient.

Single-rollout sampling can dramatically improve the stability and effectiveness of RL training for large language models, outperforming traditional methods by a significant margin.

TurnOPD redefines on-policy distillation by optimizing training budgets at the turn level, leading to superior agent performance without increasing training time.

Coding agents struggle with native language tasks, achieving only 78.7% resolution in a benchmark designed to reflect real customer requests in Russian.

A single minimal infinite-state tool can elevate finite-precision models to Turing completeness, while finite-state tools add virtually no expressivity.

CurateEvo transforms data curation from a static process into a dynamic, failure-driven evolution, significantly boosting performance and efficiency in LLM training.

Knowledge Debt is a silent threat to developer expertise, but it can be mitigated by designing AI agents that actively promote incidental learning.

DebugTracker reveals that understanding the debugging process can significantly enhance educational assessments, moving beyond just final code quality to the intricacies of student reasoning and problem-solving.

CXI ensures that language-model agents execute tasks only when all authority checks align, achieving unprecedented security with zero unauthorized escapes.

Concealing payloads using Unicode's TAG block allows attackers to inject malicious metadata into models without detection, undermining client-side defenses.

Resolve rates mask critical insights about coding agent performance, but TraceProbe uncovers the hidden trajectory structures that explain why some runs succeed while others fail.

Navigation success alone is a poor predictor of real-world web agent effectiveness, as revealed by the extensive evaluation of WebRetriever's diverse protocols.

Robots can now achieve over double the success rate in using novel tools by leveraging keypoint trajectory reasoning for functional generalization.

Emerging risks from agentic AI demand urgent interdisciplinary collaboration to secure our future.

SageMath integration boosts LLM performance in solving advanced mathematical problems by nearly 10 percentage points, narrowing the gap between open and closed models.

Evaluating code agents through their entire interaction trajectory reveals critical insights that traditional benchmarks overlook.

Agentic code review can transform AI-generated pull requests into high-quality solutions, outperforming traditional methods in both accuracy and usefulness.

Balancing automation and human intervention can significantly enhance efficiency in service systems, with the UCB-DPP policy proving to be a game-changer in managing this tradeoff.

A zero-shot LLM can match the classification accuracy of a supervised ML classifier in cryogenic fault diagnosis with just six labeled demonstrations, revolutionizing how we approach fault detection in quantum computing.

Early detection of failure in LLM agents can save over 47% of inference compute by leveraging internal representations rather than observable behavior.

The framework reveals that optimizing reasoning effort can significantly impact both the cost and complexity of model discovery, challenging conventional benchmarks.

TopoBrick achieves superior zero-shot forecasting by intelligently sampling exogenous variables based on building topology, outperforming traditional models without the need for extensive training.

SkillReranker redefines skill selection by leveraging semantic decomposition to improve task performance and efficiency in agent systems.

TOFFEE can synthesize high-quality data agent trajectories that significantly improve LLM performance in unfamiliar analytical workflows.

Curiosity in AI isn't static; it evolves based on experience, reshaping how agents prioritize questions and explore knowledge landscapes.

Current LLMs falter in complex deliberative collaboration tasks, revealing critical gaps in their reasoning capabilities even when aided by external tools.

Developers improved their prompt engineering skills significantly after just one hour with Prompt Coach, a tutor that adapts to their coding context.

AgoraSim reveals how hybrid agent-based modeling can transform LLM outputs into actionable insights for social scenario analysis.

Autonomous AI can now manage complex IPoDWDM networks, significantly improving lifecycle automation and control.

State-of-the-art LLM agents face a staggering performance decline in multilingual workflows, revealing critical gaps in current evaluation methods.

Achieving seamless automation in multi-vendor IPoDWDM networks could revolutionize how we manage and optimize complex network infrastructures.

Agents in PCBWorld can achieve near rule-based performance in PCB routing, showcasing the power of interactive, engine-grounded learning.

StateFuse reveals that preserving contradictions in memory can enhance safety and correction mechanisms in multi-agent systems, challenging the conventional wisdom of collapsing conflicting information for simplicity.

Search-augmented LLMs can achieve up to a 16% improvement in task success by learning when to avoid unnecessary searches.

A staggering 69% to 98% failure rate in real denylist policy enforcement highlights a critical vulnerability in AI coding agents that remains largely unaddressed.

LogicHunter uncovers 40 hidden bugs in LLM agent frameworks that traditional testing methods missed, achieving a groundbreaking 91.17% precision in bug detection.

AgentTether repairs over 65% of failures in complex LLM tasks without modifying the agent, revolutionizing how we ensure reliability in AI deployments.

TypeGo slashes planning delays by over 50% for embodied agents, enabling real-time responsiveness in complex environments.

AI coding assistants are reshaping open-source development by increasing contributor activity while simultaneously raising concerns about code maintainability.

CanvasAgent can adapt its tool decisions in real-time, significantly improving the quality and coherence of complex image creations.

Light-Omni achieves a remarkable 12.1× speedup in video understanding while enhancing accuracy, redefining efficiency in agentic video processing.

GaP outperforms traditional methods in variational automation tasks by leveraging directed computation graphs for real-time adaptability and improved success rates.

Recursive self-improvement in LLM agents can yield up to 23.54 points in accuracy, transforming how we approach skill evolution in AI.

Cortex outperforms traditional models by enabling zero-shot execution of complex long-horizon tasks, bridging the gap between high-level planning and low-level execution.

CompactionRL enables LLMs to effectively manage long-horizon tasks by summarizing context, leading to substantial performance gains in coding benchmarks.

Web agents can now safely interact with complex environments while avoiding prompt injection attacks by masking untrusted content without ever reading it.

PDEFlow automates the entire process from user input to solver-free inference, enabling rapid experimentation with complex differential equations.

RL-Ballast reduces decision-making steps by nearly a third while achieving 100% accuracy in identifying blockage candidates under limited sensor conditions.

GSS reveals that by sharing sampled futures, we can dramatically reduce the computational burden of planning in continuous spaces, breaking the exponential curse of horizon dependence.

Full-sovereign scaffolding not only boosts user sovereignty scores but also curtails privacy violations and manipulative behaviors in personal agents.

ReOPD turns the costly process of agent-environment interaction into a reusable offline resource, achieving faster training while preserving accuracy.

Ability-guided transfer reveals that agents struggle to consistently leverage learned experiences, challenging the effectiveness of current self-evolution methods.

Even state-of-the-art language models struggle significantly in real-world tasks, exposing critical shortcomings in their deployment readiness.