Search papers, labs, and topics across Lattice.
This paper introduces SkillSV, a structure-aware Shapley-style framework designed to evaluate the internal value of agent skills by analyzing their structured components such as rules and heuristics. By employing a method that separates content value from context cost, SkillSV provides insights into the interactions and dependencies among skill units, which is crucial for optimizing agent performance. The evaluation on four benchmarks demonstrates that SkillSV effectively recovers unit interactions and supports safe skill pruning and compression while maintaining overall skill efficacy.
SkillSV reveals the intricate value of agent skills, enabling precise optimization and safe pruning of complex skill structures.
Agent skills are increasingly optimized by automated feedback loops, producing long structured artifacts whose internal value remains unclear. We study skill valuation: assigning credit to the internal units of a fixed skill, such as rules, examples, scripts, and heuristics, under a fixed agent and held-out task distribution. Skill valuation differs from data or prompt-span valuation because skill units are structured: they may depend on other units, belong to a document hierarchy, trigger agent behavior, and consume limited prompt context. We introduce SkillSV, a structure-aware Shapley-style framework for skill valuation. SkillSV compiles a skill into units, dependencies, and hierarchy, so that only valid counterfactual skills are evaluated. It uses paired deletion and length-neutral padding to separate content value from context cost, and estimates the resulting values with a rollout-budgeted estimator for noisy agent evaluations. On four agentic benchmarks, we assess the faithfulness, actionability, and explanation of SkillSV: it recovers unit interactions, preserves aggregate skill lift, and guides safe pruning and compression.
