Search papers, labs, and topics across Lattice.
This paper introduces TEE-X, an acceleration framework designed to optimize large vision models within Trusted Execution Environments (TEEs) for edge applications. By employing a sensitivity-aware modularization technique and enabling vectorization, TEE-X addresses the challenges of memory constraints and computational latency, achieving GPU-level inference speeds. The results demonstrate that TEE-X maintains high performance with minimal accuracy-latency trade-offs, significantly enhancing security and privacy for time-sensitive vision tasks.
TEE-X achieves GPU-level inference latency for large vision models while ensuring robust security in edge applications, transforming the landscape of model deployment.
Despite their remarkable success, machine learning models, particularly in vision applications, are alarmingly vulnerable to a range of security threats. One key factor in the attack landscape is the distinction between white-box and black-box threat models, as the latter poses challenges that limit attack effectiveness when access to model information is limited. As a result, using Trusted Execution Environments (TEEs) enhances security for machine learning applications by protecting model confidentiality and execution integrity, effectively shifting the execution environment from the white-box to the black-box side of the threat model spectrum. While adopting TEEs for large vision models, e.g., Vision Transformers (ViTs), is crucial for enhancing security and privacy, significant challenges related to memory constraints and increased computational latency must be addressed, especially in time-sensitive edge applications where safety and privacy are paramount. The objective of this work is to enable large vision models to be fully hosted within TEEs, achieving GPU-level inference latency for time-sensitive edge vision applications while maintaining performance. To this end, we propose TEE-X, a TEE-aware acceleration framework that introduces a sensitivity-aware modularization technique and enables vectorization in TEE inference. This design is validated on OP-TEE for Arm TrustZone, configured to optimize performance on the NVIDIA Jetson AGX Xavier for efficient edge vision applications using ViT models. The findings reveal that TEE-X delivers an effective TEE-aware acceleration framework that achieves minimal accuracy-latency trade-offs while ensuring fast and secure edge inference for vision models.
