Search papers, labs, and topics across Lattice.
The paper introduces MOSAIC, a novel framework for designing heterogeneous neural processing units (HPUs) that optimally balance various architectural dimensions, including tile types and dataflows, to accommodate the diverse computational needs of emerging AI model architectures. By employing a multi-seed pipeline that combines stratified sweeping and genetic-algorithm refinement, MOSAIC effectively identifies Pareto-optimal designs tailored for a wide range of workloads. The results show that the best general-purpose HPU configuration achieves a remarkable 46.91% mean iso-area energy savings compared to the best homogeneous baseline, highlighting the efficiency gains possible with fine-grained tile-level heterogeneity.
MOSAIC uncovers a 46.91% energy savings in heterogeneous NPUs, challenging the dominance of homogeneous designs in AI workloads.
AI model architectures are diversifying rapidly. Although dense matrix multiplication underlies today's CNNs and transformers, emerging architectures (state-space models, long convolutions via the fast Fourier transform (FFT), Kolmogorov-Arnold networks, and spiking networks) are not multiply-accumulate (MAC) dominated; they spend much of their computation on vector and non-MAC primitives that homogeneous, MAC-centric neural processing units (NPUs) serve poorly. This has motivated heterogeneous NPUs (HPUs) built from non-identical tiles. Prior heterogeneous designs vary only one or two coarse knobs (typically MAC precision or array size) and are evaluated on narrow workloads; no existing framework supports fine-grained HPU design, where tiles differ across many architectural dimensions at once. We present MOSAIC, an analytical simulator and design-space-exploration (DSE) framework for HPU microarchitecture design. MOSAIC searches the joint space of tile-level heterogeneity: beyond array size and precision, it varies tile-type composition (large Big, small Little, and non-MAC Special-Function tiles), dataflow, sparsity mode, MAC engine type, and special-function units for non-MAC operators (FFT, spiking-integrate, polynomial). Unlike prior simulators that model a single homogeneous tile type, MOSAIC models non-MAC tiles with their own energy, area, and timing models and maps operators across a mix of tiles with a heterogeneity-aware compiler. A multi-seed pipeline pairing a stratified sweep with genetic-algorithm refinement returns Pareto-optimal designs, with cost models calibrated to a 7 nm node and cross-validated against NVIDIA's Deep Learning Accelerator (NVDLA). Across a 20-workload suite, the best general-purpose HPU found by MOSAIC (~200 mm^2 Big+Little+Special-Function) achieves +46.91% mean iso-area energy savings over the best iso-area homogeneous baseline.
