Search papers, labs, and topics across Lattice.
This paper conducts a systematic security analysis of AI-powered applications (AI-Apps) hosted on major platforms like Hugging Face, revealing critical vulnerabilities that arise from weak isolation and misconfigured security settings. By mapping the AI-App lifecycle to established risk taxonomies, the authors identify five threat categories and ten attack vectors, uncovering issues such as broken access control and insecure resource reuse. The analysis, applied to over 970,000 public AI-Apps, highlights alarming findings, including thousands of apps leaking credentials and hundreds with input injection vulnerabilities, underscoring the urgent need for improved security measures in this rapidly growing ecosystem.
Thousands of AI-Apps are leaking sensitive credentials and harboring vulnerabilities that could lead to arbitrary code execution, revealing a critical security crisis in the AI ecosystem.
AI-powered Applications (AI-Apps), hosted on platforms such as Hugging Face, are democratizing access to pre-trained models through online inference and fine-tuning services. While lowering AI adoption barriers, these platforms introduce an unexplored attack surface, as AI-Apps are often developed by untrusted parties with weak isolation and misconfigured security settings. In this paper, we present the first systematic security analysis of AI-Apps across three leading platforms. To structure our investigation, we map the AI-App lifecycle to established risk taxonomies (e.g., OWASP), identifying five threat categories and ten attack vectors ranging from generic web flaws to high-impact architectural issues. Our analysis reveals critical failures including broken access control, insecure resource reuse, insufficient input validation, and sensitive data exposure. Notably, we uncover three novel architectural vulnerabilities inherent to platform design and demonstrate how traditional issues (e.g., world-readable logs) are uniquely amplified in this ecosystem. To assess real-world impact, we develop an analysis framework Insightor and apply it to over 970,000 public AI-Apps. Alarmingly, we find thousands of apps leaking credentials, hundreds containing input injection vulnerabilities that allow arbitrary code execution, and tens harboring embedded backdoors -- indicating active exploitation. We have responsibly disclosed all findings to the affected platforms and developers.
Tsinghua AI