Search papers, labs, and topics across Lattice.
This paper introduces CloakLM, a software-only framework designed to obfuscate GPU memory layouts in order to mitigate model exfiltration risks associated with large foundation models deployed on shared infrastructure. By employing techniques such as PCIe traffic shaping, weight shuffling, and HBM page remapping, CloakLM effectively disrupts the structural regularity of model weights without altering the logical memory view for authorized users. Evaluation results indicate that CloakLM maintains near-native performance while significantly enhancing resistance to attacks like PCIe snooping and HBM dumps, thus making inference-time model exfiltration considerably more challenging.
CloakLM makes inference-time model exfiltration substantially less practical by obfuscating memory layouts without compromising performance.
Large foundation models deployed on third-party and shared accelerator infrastructure face a practical risk of model exfiltration that existing defenses do not fully address. In common serving deployments, model providers control the VM or bare-metal serving stack but not the surrounding hardware substrate. The host to GPU interconnect, accelerator fabric, and neighboring infrastructure components remain outside the tenant's trust boundary and have been shown to be exploitable. Hermes demonstrates lossless DNN reconstruction from passive PCIe observation, while TunnelS exfiltrates HBM contents at high throughput via driver-level access without disrupting inference. Co-tenant VMs can further access memory-mapped interfaces or misconfigured RDMA regions without physical co-location. These attacks exploit a common property of ML systems: model weights are stored in large, contiguous, and repeatedly accessed memory regions, making intercepted PCIe transfers and HBM dumps rich enough to reveal model structure and parameters. We present CloakLM, a software-only memory-obfuscation framework that removes this structural regularity without changing the inference stack's logical view of memory. CloakLM combines three mechanisms: PCIe traffic shaping, inter- and intra-layer weight shuffling, and physical HBM page remapping. Authorized execution retains a valid virtual memory layout with negligible overhead, while unauthorized observers see fragmented and semantically incoherent state. CloakLM integrates with vLLM and PyTorch, requires no hardware changes, and complements confidential computing. Evaluation on distributed inference workloads using LLaMA and Qwen models shows near-native performance while significantly increasing resistance to PCIe snooping and HBM dump attacks, making inference-time model exfiltration substantially less practical.
