Search papers, labs, and topics across Lattice.
This paper introduces AntiSkillBench, a comprehensive benchmark designed to evaluate privacy leakage and impersonation risks in persona skills, which are increasingly used to personalize agent interactions. By analyzing a dataset of 7,500 dialogue traces from diverse profiles and testing various distillation strategies, the study reveals that risks of attribute disclosure and behavioral impersonation are prevalent across different agent architectures. The findings indicate that current defenses are largely ineffective, underscoring the need for improved privacy-preserving techniques in the development of persona skills.
Persona skills expose significant privacy risks, with existing defenses failing to adequately protect against attribute disclosure and impersonation across diverse agent architectures.
Persona skills distill personal interaction histories into portable and executable artifacts for downstream agents. While enabling flexible personalization, this process concentrates fragmented personal signals, amplifies their impact through reuse, and challenges defenses designed for individual records or retrieval-based memory. To systematically investigate the safety of the persona-skill pipeline, we introduce AntiSkillBench, an end-to-end benchmark for evaluating risks and defenses across the persona-skill pipeline. It comprises: (i) a dataset of 7,500 persona-grounded dialogue traces, constructed from 50 behaviorally rich profiles spanning diverse task scenarios; (ii) an evaluation suite that measures skill-level privacy leakage and agent-level attribute disclosure and behavioral impersonation across three skill-distillation strategies; and (iii) a defense evaluation covering four configurations across online and post-hoc interventions, including active risk suppression and passive provenance protection. Experiments across three frontier agents show that persona-skill risks persist across agent backbones and distillation protocols, extending from explicit attributes to communication styles and personality traits. Existing defenses exhibit limited and distillation-dependent effectiveness, failing to generalize across risk and distillation strategies. These results highlight AntiSkillBench as a challenging benchmark for developing privacy-preserving and authenticity-aware persona skills.
