Search papers, labs, and topics across Lattice.
This paper introduces Fiber Memory, an innovative architecture that utilizes optical fiber as an active, recirculating delay-line memory specifically designed for immutable data, such as large language model weights. By leveraging space-division multiplexed multi-core fibers and advanced optical interfaces, the proposed system significantly reduces the energy consumption associated with weight delivery and eliminates redundant storage across thousands of AI accelerators. The results indicate a more than 70% reduction in weight-delivery energy compared to conventional high-bandwidth memory configurations, addressing the growing demand for memory in generative AI applications.
Fiber Memory can slash weight-delivery energy by over 70% while eliminating redundant storage across thousands of AI accelerators.
The rising pressure on DRAM availability and contract pricing reflects generative AI's massive high-performance memory requirements. This pressure is heavily compounded by hyperscale data center expansion, which now consumes a significant portion of global DRAM output. In this work, we propose a new architecture: Fiber Memory, which reimagines the role of optical fiber in a hyperscale data center, deploying it as an active, recirculating delay-line memory for immutable data, such as large language model (LLM) weights. We present a data-parallel optical broadcast delay-line memory architecture that accounts for fiber's physical realities. By incorporating space-division multiplexed multi-core fibers (MCFs), passive optical tap-and-amplify interfaces, co-packaged optics (CPO), and regional all-optical regeneration, our case study evaluation demonstrates that Fiber Memory can eliminate redundant weight storage across 10,000 AI accelerators and reduce weight-delivery energy by over 70% compared to traditional HBM3e configurations.
