Search papers, labs, and topics across Lattice.
This paper explores network topologies for wafer-scale systems using wafer-on-wafer hybrid bonding to address communication bottlenecks in large language models. Four novel reticle placements (Aligned, Interleaved, Rotated, and Contoured) are proposed and compared against a 2D mesh baseline, evaluating their impact on throughput, latency, and energy efficiency. Results demonstrate that optimized reticle placements can achieve up to 250% higher throughput, 36% lower latency, and 38% lower energy consumption per byte transmitted.
Clever reticle placement on wafer-scale systems can boost throughput by 2.5x and slash latency by over a third, offering a hardware-level speedup for LLM training.
Transformer-based large language models are increasingly constrained by data movement as communication bandwidth drops sharply beyond the chip boundary. Wafer-scale integration using wafer-on-wafer hybrid bonding alleviates this limitation by providing ultra-high bandwidth between reticles on bonded wafers. In this paper, we investigate how the physical placement of reticles on wafers influences the achievable network topology and the resulting communication performance. Starting from a 2D mesh-like baseline, we propose four reticle placements (Aligned, Interleaved, Rotated, and Contoured) that improve throughput by up to 250%, reduce latency by up to 36%, and decrease energy per transmitted byte by up to 38%.
ETH