Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
DetShare is a GPU sharing system designed to provide both semantic and performance determinism without requiring kernel modifications. It introduces GPU coroutines to decouple logical execution contexts from physical resources, enabling fine-grained resource allocation through lightweight context migration. Evaluation shows DetShare improves training throughput, reduces tail latency in co-location scenarios, and decreases average inference latency compared to existing methods.
Achieve near-ideal GPU sharing without kernel hacks: DetShare guarantees semantic and performance determinism through GPU coroutines and lightweight context migration.
GPU sharing is critical for maximizing hardware utilization in modern data centers. However, existing approaches present a stark trade-off: coarse-grained temporal multiplexing incurs severe tail-latency spikes for interactive services, while fine-grained spatial partitioning often necessitates invasive kernel modifications that compromise behavioral equivalence. We present DetShare, a novel GPU sharing system that prioritizes determinism and transparency. DetShare ensures semantic determinism (unmodified kernels yield identical results) and performance determinism (predictable tail latency), all while maintaining complete transparency (zero code modification). DetShare introduces GPU coroutines, a new abstraction that decouples logical execution contexts from physical GPU resources. This decoupling enables flexible, fine-grained resource allocation via lightweight context migration. Our evaluation demonstrates that DetShare improves training throughput by up to 79.2% compared to temporal sharing. In co-location scenarios, it outperforms state-of-the-art baselines, reducing P99 tail latency by 15.1% without compromising throughput. Furthermore, through workload-aware placement and our TPOT-First scheduling policy, DetShare decreases average inference latency by 69.1% and reduces Time-Per-Output-Token (TPOT) SLO violations by 21.2% relative to default policies.