Search papers, labs, and topics across Lattice.
This paper introduces AgenticCANN, a novel framework for automated Ascend C operator synthesis that leverages knowledge-augmented agentic evolution to address the unique challenges of NPU inference performance. By integrating a knowledge-orchestrated generation system and a stage-adaptive evolution strategy, the framework effectively bridges the gap in hardware expertise, achieving feasibility rates of 90-100% for elementwise and normalization operators and a significant speedup of up to 6.65脳 for 1B Pangu model inference kernels. The findings underscore the critical role of knowledge injection in enhancing operator feasibility, demonstrating a generalizable benefit across different operator types.
Achieving up to 6.65脳 speedup in NPU inference by automating Ascend C operator generation could revolutionize performance optimization in low-corpus environments.
Ascend C operator optimization is critical for NPU (Neural Processing Unit) inference performance but requires deep hardware expertise.While large language models (LLMs) have shown promise in automated CUDA kernel generation, the fundamentally different programming model of Ascend C introduces unique challenges that remain unexplored. In this paper, we propose AgenticCANN, a knowledge-augmented agentic evolution framework specifically tailored for automated Ascend C operator synthesis in low-corpus NPU environments.To overcome the severe platform knowledge deficit on unfamiliar hardware, AgenticCANN incorporates a knowledge-orchestrated generation system that delivers structured, multi-level domain insights across the development lifecycle to resolve the upstream feasibility bottleneck.Building on this foundation, it features a stage-adaptive agentic evolution strategy that dynamically aligns LLM interaction modes with specific generation and evolution phases, balancing high-exploration candidate discovery with high-convergence performance tuning.Extensive experiments on Huawei Ascend 910B across six operators spanning five pattern categories demonstrate that our method achieves 90 to 100 percent feasibility on elementwise and normalization operators, 56% on fusion operators, and up to 6.65$\times$ speedup on 1B Pangu model inference kernels. Further analysis reveals that knowledge injection monotonically improves feasibility from 57% to 86% on elementwise operators, demonstrating its general rather than operator-specific benefit.
