Search papers, labs, and topics across Lattice.
This paper investigates the vulnerability of large language model (LLM) agents that utilize reusable skills, focusing on potential skill injection attacks. By implementing guardian-based defenses鈥攂oth dynamic and static鈥攖he authors significantly reduce the attack success rate (ASR) from 81.4% to 18.6% while maintaining task performance across multiple LLM families. The findings underscore the effectiveness of real-time mediation in enhancing the security of LLM agents against skill injection threats.
Guardian-based defenses can slash skill injection attack success rates by over 75% without sacrificing task performance.
Large language model (LLM) agents increasingly rely on reusable skills i.e. documents describing task-specific procedures. However, this introduces a new attack surface for agents to manage. We study two complementary directions for this threat. First, we evaluate guardian-based defenses: an intermediary LLM agent that acts as a mediator for skill file access (dynamic guardian) or pre-rewrites these files at build time (static guardian). Across three LLM agent families, our guardians cut attack success rate (ASR) by well over half while preserving task utility. Second, we stress test them through attack reframing using four attacks that preserve the malicious instruction but change the phrasing. For non-guardian setup, the reframing pushes the ASR up to 81.4\%, but the dynamic guardian brings it down to 18.6\%, showing that real-time mediation is a robust defense.
