Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
To replace costly runtime autotuning across tens of thousands of candidate GPU kernels, this work designs a learned kernel selection framework powered by statically computable estimates of induced hardware behavior rather than raw hyperparameter configurations. Evaluating on a massive dataset of 4.9 million CUTLASS kernels, learning-to-rank models augmented with these microarchitectural features reduce selection regret by up to 40% over structural baselines and 64.2% compared to NVIDIA’s default GEMM heuristics. Crucially, explicitly representing hardware execution characteristics provides the inductive bias needed to successfully transfer across numerical precisions and epilogue fusions without retraining from scratch.
NVIDIA's hand-crafted GEMM heuristics leave massive performance on the table: grounding learned kernel selectors in static hardware-behavior estimates cuts selection regret by up to 64.2% without running an autotuning benchmark.
GPU libraries such as CUTLASS expose tens of thousands of semantically equivalent kernels for a single operation, making exhaustive autotuning expensive and execution-free selection difficult. Existing analytical selectors require hand-designed performance rules, while learned selectors operate on raw configuration parameters and must infer hardware consequences from data. We introduce a hardware-aware representation for CUTLASS kernel selection that augments candidate configurations with statically computable estimates of induced hardware behavior. We construct a dataset of 4.9 million CUTLASS kernels and train gradient-boosted and neural learning-to-rank models to rank candidates within each problem. On held-out exhaustive evaluation problems, hardware-aware representations reduce selection regret by up to 40\% relative to structural baselines and 64.2\% relative to NVIDIA's matrix-multiply heuristics. We further evaluate data-efficient cross-precision and epilogue-fusion transfer within CUTLASS GEMM, showing that explicitly representing candidate-induced hardware behavior provides a useful inductive bias for learned kernel selection.