Search papers, labs, and topics across Lattice.
This paper investigates the optimal implementation of extreme-edge scientific neural networks on AI Engines versus programmable logic within FPGA SoCs. It introduces a latency-adjusted resource equivalence (LARE) metric to guide design choices, accounting for the distinct architectural and performance characteristics of AI Engines. The authors demonstrate the successful deployment of end-to-end neural networks on AI Engines that exceed the capacity of programmable logic when using the hlsml toolchain, highlighting the benefits of AI Engines for low-latency scientific inference.
Forget struggling with resource constraints: AI Engines on FPGAs can now handle end-to-end scientific neural networks that are too large for programmable logic, unlocking new possibilities for extreme-edge applications.
Extreme-edge scientific applications use machine learning models to analyze sensor data and make real-time decisions. Their stringent latency and throughput requirements demand small batch sizes and require that model weights remain fully on-chip. Spatial dataflow implementations are common for extreme-edge applications. Spatial dataflow works well for small networks, but it fails to scale to larger models due to inherent resource scaling limitations. AI Engines on modern FPGA SoCs offer a promising alternative with high compute density and additional on-chip memory. However, the architecture, programming model, and performance-scaling behavior of AI Engines differ fundamentally from those of the programmable logic, making direct comparison non-trivial and the benefits of using AI Engines unclear. This work addresses how and when extreme-edge scientific neural networks should be implemented on AI Engines versus programmable logic. We provide systematic architectural characterization and micro-benchmarking and introduce a latency-adjusted resource equivalence (LARE) metric that identifies when AI Engine implementations outperform programmable logic designs. We further propose spatial and API-level dataflow optimizations tailored to low-latency scientific inference. Finally, we demonstrate the successful deployment of end-to-end neural networks on AI Engines that cannot fit on programmable logic when using the hlsml toolchain.
