Search papers, labs, and topics across Lattice.
This study analyzes 138,133 public SKILL.md files to identify barriers to reusability in Agent Skills for LLMs, revealing that 91.8% of skills contain defects primarily due to packaging issues rather than complex vulnerabilities. The research highlights that skills with valid routing metadata are retrieved more reliably, emphasizing the importance of proper organization and metadata in enhancing skill functionality. Additionally, the authors propose a quality-assured generation workflow that integrates specification-aware prompting and automated repair to address these defects effectively.
A staggering 91.8% of public Agent Skills are flawed, primarily due to simple packaging issues that hinder their reusability.
Under the current standard, Agent Skills are SKILL.md files that combine instructions with supporting resources, enabling Large Language Model (LLM) agents to reuse procedures beyond a single conversation. Yet many public skills appear to originate from a single task, repository, or conversation, even when they are shared as reusable components. We analyze this gap across 138,133 public SKILL.md files from 20,556 repositories using a two-tier defect taxonomy grounded in the official specification and best-practice guidance. We find that 91.8% of skills contain at least one detected defect, with stable estimates across lenient and strict thresholds (88.8-94.6%). The dominant failures are ordinary packaging problems rather than exotic attacks: weak routing metadata, bloated or non-actionable bodies, and poor resource organization. A deterministic routing stress test over 20,000 skills shows the functional impact: skills with valid routing metadata are retrieved more reliably from startup descriptions than skills with routing defects. Defect rates vary by platform and provenance: specification-aware skills contain fewer defects, while AI-marked skills show more safety and portability problems. Lightweight enforcement and repair experiments support a quality-assured generation workflow combining spec-aware prompting, lightweight linting, automated repair, and safety gating.
