Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces an RL-driven compiler that co-optimizes ASIC architecture, memory hierarchy, and workload partitioning for on-device AI inference. The design space is modeled as a Markov Decision Process with a unified Power-Performance-Area (PPA) objective, and solved using Soft Actor-Critic (SAC) with Mixture-of-Experts gating. Validated on Llama 3.1 8B and SmolVLM across 7 process nodes, the approach automatically adapts mesh sizes and per-tile configurations, achieving 29809 tokens per second at 3nm and less than 13 mW at 10 MHz.
Forget hand-tuning: RL can automatically adapt ASIC architectures for on-device LLM inference across diverse process nodes, optimizing for power, performance, and area.
We present an RL-driven compiler that jointly optimizes ASIC architecture, memory hierarchy, and workload partitioning for AI inference across 3nm to 28nm. The design space is formulated as a single Markov Decision Process with mixed discrete-continuous actions and a unified Power-Performance-Area (PPA) objective. Soft Actor-Critic (SAC) with Mixture-of-Experts gating explores the joint space of mesh topology, per-core microarchitecture, and operator placement. We validate on two workloads, Llama 3.1 8B FP16 (high-performance mode, 29809 tokens per second at 3nm) and SmolVLM (low-power mode, less than 13 mW at all nodes, 10 MHz). Across 7 process nodes, the RL automatically adapts mesh sizes and per-tile configurations, including heterogeneous FETCH, VLEN, and memory allocation without node-specific manual retuning.