Search papers, labs, and topics across Lattice.
This paper introduces FIBER, a novel architecture that decouples the execution of tensor computations from private register ownership in GPUs, addressing the inefficiencies caused by fixed parallelism and coarse-grained scheduling in modern AI workloads. By implementing a shared-register addressing scheme and fine-grained scheduling, FIBER significantly enhances dynamic parallelism and reduces redundancy in operand supply. The results demonstrate remarkable performance improvements, achieving up to 2.25x speedup in end-to-end processing for mixed-precision LLM serving on Ampere, and even higher gains on Hopper and Blackwell architectures.
FIBER achieves up to 2.25x speedup in LLM serving by decoupling tensor computation from register ownership, revolutionizing GPU efficiency.
Modern GPUs increasingly integrate Tensor Cores into the execution pipeline. Although aggregate tensor throughput continues to grow, aided by an operand supply that has evolved from register-based in Ampere to redundancy-free, memory-based in Hopper and Blackwell, efficiently orchestrating the complete tensor compute pipeline for the modern AI workloads remains challenging. We identify the fundamental bottlenecks as fixed parallelism and coarse-grained scheduling, both of which are exposed by modern AI workloads that interleave diverse non-GEMM operations with GEMM. To orchestrate tensor computation efficiently, we propose FIBER, a new architecture that extends the GPU SIMT (single instruction, multiple thread) model. Its basic execution instance, the \emph{fiber}, is decoupled from private register ownership, carrying only minimal control state while accessing an SM's registers through a shared view. This enables dynamic parallelism scaling, fine-grained register-level dataflow scheduling, and offers a redundancy-free alternative for matrix operand supply. We extend the ISA, microarchitecture, and compiler to realize shared-register addressing, conflict-free operand delivery, and fiber-based program mapping. Under a typical mixed-precision LLM serving scenario, FIBER achieves a 2.25x end-to-end speedup on Ampere (1.15x for the original FP16 computation), with 1.8x and 2.09x on Hopper and Blackwell respectively, and kernel-level gains up to 2.49x.
