Search papers, labs, and topics across Lattice.
This study evaluates the warp divergence handling of NVIDIA GPUs across different architectures, including Ampere, Hopper, and Blackwell, using Pascal as a baseline. By employing cycle-accurate microbenchmarks, hardware counters, and static analysis, the authors reveal that the performance cost associated with divergent paths remains stable and predictable, despite architectural changes in reconvergence mechanisms. Notably, the findings indicate that while the compiler's reconvergence strategies have evolved significantly, the fundamental performance penalties of warp divergence have not, suggesting a consistent programmer-visible cost model across generations.
Warp divergence incurs a predictable performance cost across NVIDIA GPU architectures, even as reconvergence mechanisms evolve dramatically.
Since Volta introduced Independent Thread Scheduling (ITS), NVIDIA GPUs have been widely assumed to handle warp divergence in a fixed manner. We test this assumption across Ampere, Hopper, and datacenter and consumer Blackwell GPUs, using pre-ITS Pascal as a baseline. Combining cycle-accurate microbenchmarks, hardware counters, and static analysis of compiler-generated SASS, we separate stable behavior from architectural change. Across all tested generations, divergent paths serialize linearly with the number of paths $k$, following $T(k) \approx sk$ with no super-linear reconvergence penalty. Warp execution efficiency falls as $32/k$, the penalty is independent of occupancy, and predication removes the serialization cost. The same behavior appears on Pascal, showing that this programmer-visible cost model predates ITS. The compiler-emitted reconvergence machinery, however, has changed substantially. Pascal uses a per-warp SSY/SYNC instruction stack, whereas later generations use barrier-register instructions. Deferred reconvergence beyond the immediate post-dominator falls from 29 cases on Ampere to 2 on Blackwell. Blackwell also introduces a two-tier convergence-barrier classification, uniform-branch instructions, and explicit partial-mask warp synchronization, none of which appear on Ampere or Hopper. Controlled bit-flip experiments indicate that the new barrier class is a static compiler classification with no observable runtime effect in our tests. Thus, divergence retains a stable and predictable performance cost even as NVIDIA's control-flow ISA and reconvergence mechanisms continue to evolve.
