Search papers, labs, and topics across Lattice.
This paper investigates the performance degradation of Large Language Model (LLM) serving under GPU Confidential Computing (GPU-CC) on Blackwell platforms, revealing that the primary bottleneck arises from the serialized VM-GPU bridge rather than GPU compute itself. The authors quantify the throughput loss, which ranges from 13-27%, and demonstrate that the inefficiencies in host/device movement and KV-cache restore latency can significantly hinder performance. By implementing targeted scheduling flags and worker-thread optimizations, they achieve up to a 92% recovery in throughput, highlighting critical areas for improvement in confidential AI infrastructures.
The performance gap in LLM serving under GPU-CC is primarily due to a costly serialized VM-GPU bridge, not the GPU's computational capabilities.
GPU Confidential Computing (GPU-CC) now preserves GPU-local performance: on NVIDIA B300, BF16 matmul runs at 0.998x of non-confidential performance. Yet LLM serving under Intel TDX plus GPU-CC still loses 13-27% of throughput, and KV-cache restore latency can more than double. This paper studies that gap on two Blackwell platforms, RTX Pro 6000 and B300 HGX, and identifies its dominant cause: the confidential VM-GPU bridge, not GPU compute. We find that GPU-CC turns host/device movement into a serialized, high-setup-cost channel. Secure copies do not gain CUDA-stream concurrency within a context, asynchronous transfers block at the runtime boundary, and small crossings pay a fixed toll. This violates the assumptions of modern inference runtimes, where DMA is expected to be cheap, concurrent, and asynchronous. In vLLM dense decode, the gap closes around 44x-slower small alloc-and-copy operations; targeted patches reject alternative explanations. A scheduling flag recovers 57% of the gap, while a worker-thread drain recovers up to 92% in qualified high-concurrency runs. The same bridge model explains a +131% KV-restore penalty and a 34x model-load slowdown. Blackwell also changes the confidential tenancy unit. We qualify confidential multi-GPU NVSwitch tenants on B300, including 510 GB/s NVLink P2P inside a CVM and concurrent isolated tenants, and identify the remaining fabric-attestation gap for production confidential AI platforms.
