Track synthesis, materials and catalysis as it is published.
Search papers, labs, and topics across Lattice.
This paper introduces a heterogeneous analog-digital computing framework for Mixture-of-Experts (MoE) models that avoids noise-aware retraining by strategically assigning noise-sensitive experts and densely activated modules to digital computation. The approach identifies noise-sensitive experts based on their maximum neuron norm, enabling the majority of experts to be executed on analog in-memory computing (AIMC) hardware. Experiments on DeepSeekMoE and OLMoE demonstrate the framework's robustness in maintaining accuracy under analog nonidealities across various benchmark tasks.
Achieve robust MoE inference on analog hardware without retraining by intelligently offloading only the most noise-sensitive experts to digital compute.
Sparse Mixture-of-Experts (MoE) models enable efficient scalability by activating only a small sub-set of experts per input, yet their massive parameter counts lead to substantial memory and energy inefficiency during inference. Analog in-memory computing (AIMC) offers a promising solution by eliminating frequent data movement between memory and compute units. However, mitigating hardware nonidealities of AIMC typically requires noise-aware retraining, which is infeasible for large MoE models. In this paper, we propose a retraining-free heterogeneous computation framework in which noise-sensitive experts, which are provably identifiable by their maximum neuron norm, are computed digitally while the majority of the experts are executed on AIMC hardware. We further assign densely activated modules, such as attention layers, to digital computation due to their high noise sensitivity despite comprising a small fraction of parameters. Extensive experiments on large MoE language models, including DeepSeekMoE and OLMoE, across multiple benchmark tasks validate the robustness of our approach in maintaining accuracy under analog nonidealities.