Search papers, labs, and topics across Lattice.
This study investigates the detrimental effects of agent skills on the performance of LLM agents, revealing that while skills are intended to enhance task execution, they can lead to significant functional failures and efficiency regressions. By employing a differential analysis framework, the authors attribute 307 skill-induced failures to specific skills, highlighting that relevant skills often lead to incorrect task implementations rather than irrelevant ones. Key findings indicate that excessive verification and complex implementation processes are primary contributors to these failures, prompting recommendations for safer and more efficient skill usage in LLMs.
Skills that are meant to enhance LLM agents can paradoxically lead to significant task failures and inefficiencies, challenging the assumption that more skills always improve performance.
Agent skills are the de facto mechanism for extending LLM agents with reusable guidance. A skill can shape the agent's task execution, including planning, tool use, problem-solving, and validation. Prior work reported mixed results of agent skills: some skills improve task success rates, while others have no effect, increase token use and execution time, and even reduce success rates. This paper presents a comprehensive analysis of skill-induced agent failures by attributing task failures and cost regressions to specific loaded skills. We introduce a differential analysis framework that attributes a failure or regression to a skill by comparing a target skill-guided run against a no-skill or semantically matched skill reference run that solves the same task, or solves it more cheaply. We instantiate this framework on SkillsBench and SWE-Skills-Bench, yielding 307 skill-induced failures, including 125 functional failures and 182 efficiency regressions. We also build SkillTriage, a taxonomy-guided attribution tool that normalizes paired cases, extracts differential evidence, and produces triage reports. Our major findings include: (1) Skill induced functional failures are rarely caused by obviously irrelevant skills; instead, seemingly relevant skills often make the agent incorrectly implement or omit task-required implementation elements. (2) Skill-induced efficiency regressions are not explained by prompt length alone. (3) The largest sources within Excessive Procedure are excessive verification and heavy implementation pipelines, contributing 67 and 30 cases, respectively. This shows that skills often turn validation checklists and construction recipes into mandatory work. Based on our findings, we propose research topics and tooling improvements for safer and more cost-aware skill reuse.
