Search papers, labs, and topics across Lattice.
This paper introduces SAMD, an automated tool that applies System Theoretic Process Analysis for Security (STPA-Sec) to AI/ML-enabled medical devices to identify false data injection vulnerabilities. SAMD models the medical system as a control structure and uses LLMs and vulnerability databases to discover vulnerabilities and generate attack scenarios. Case studies on five FDA-cleared devices demonstrate SAMD's high precision (95.3% accuracy) in generating relevant attack scenarios, highlighting potential risks of misdiagnosis and incorrect treatments due to data injection.
Medical devices are surprisingly vulnerable: a new tool finds that adversaries can inject false data into AI/ML components with high accuracy, potentially leading to misdiagnosis and wrong treatments.
The growing integration of artificial intelligence (AI) and machine learning (ML) in medical systems requires effective measures to address emerging security risks. One such risk is that of adversaries introducing false data through vulnerable system components during inference, causing misdiagnosis and wrong treatments. These risks are challenging to anticipate and address in the design phase, as the system assembly partially occurs during actual use by end users. To address this concern, we introduce SAMD, an automated tool for performing System Theoretic Process Analysis for Security (STPA-Sec) on AI/ML-enabled medical devices during the design phase. SAMD models the medical system as a control structure, treating all system components as potential points for injecting false data into the ML engine. It leverages state-of-the-art vulnerability databases and Large Language Models (LLMs) to automate vulnerability discovery and generate a list of potential attack scenarios. We demonstrate SAMD's effectiveness through case studies on five FDA-cleared medical devices, showcasing its ability to identify vulnerable points and potential attack paths. We find that SAMD has 100% precision in identifying target device technologies in the case studies'documents, retrieves the known vulnerabilities linked to them (with 63.2% precision), and generates highly relevant attack scenarios on the ML model, including detailed steps that an adversary might take (with 95.3% accuracy, and the highest time taken being 191.64s).
