Search papers, labs, and topics across Lattice.
This paper introduces Progressive Loading-Aware Hierarchical Contrastive Learning (PL-HCL), a framework designed to detect cross-layer misalignment in Agent Skills used by large language model (LLM) agents. By leveraging a dataset of over 264,000 open-source skills and a human-verified challenge set, PL-HCL significantly enhances the detection accuracy, achieving a Macro-F1 score improvement from approximately 0.45 to 0.87-0.89 across various LLM backbones. This advancement is crucial for ensuring that users can reliably select and utilize third-party skills without encountering discrepancies between their descriptions and actual behaviors.
PL-HCL transforms the detection of inconsistencies in Agent Skills, boosting accuracy from 45% to nearly 90% and enhancing user trust in skill selection.
Large language model (LLM) agents are increasingly extended through Agent Skills, reusable artifacts that package natural-language metadata, procedural instructions, and execution-time resources for runtime use. As open-source skill marketplaces expand, users and agents increasingly rely on brief metadata to select third-party skills, making it difficult to detect inconsistencies between a skill's description and its true behavior, a problem we call cross-layer misalignment. To address this issue, we propose Progressive Loading-Aware Hierarchical Contrastive Learning (PL-HCL), an LLM-based framework that detects misalignment by modeling the layered structure of Agent Skills and learning cross-layer consistency. Using a normalized corpus of over 264,000 open-source skills and a human-verified challenge set, PL-HCL improves Macro-F1 from approximately 0.45 for unadapted baselines to 0.87-0.89 across evaluated LLM backbones. This approach offers an effective screening tool for users and operators, as well as design principles for detecting inconsistencies in layered digital artifacts.
