Search papers, labs, and topics across Lattice.
This paper introduces Ventaglio, a runtime-configurable sparse execution unit designed to optimize sparse tensor contractions on vector processors, specifically addressing the limitations of existing RVV architectures. By incorporating indexed gather-accumulate-scatter support, Ventaglio enables significant performance improvements, achieving speedups of 6.9 to 7.4 times over optimized RVV baselines with minimal area overhead. The integration of this unit into a multi-cluster system demonstrates its effectiveness in accelerating inference for pruned Transformer models, yielding up to 5.25 times faster decoding speeds compared to dense baselines.
Ventaglio accelerates sparse tensor contractions by up to 7.4 times, pushing performance limits closer to theoretical roofline bounds.
Fine-grained weight pruning and activation sparsification have emerged as effective approaches for reducing the compute and memory cost of inference for Transformer models. In the moderate-sparsity regime, Gustavson's dataflow provides a natural execution model for exploiting both activation and weight sparsity on vector processors through metadata-driven indexed accumulation. However, existing RVV architectures lack native support for this pattern, forcing kernels to rely on software index decoding and L1-backed indexed memory operations that keep sparse tensor contractions far below their roofline performance bound. We present Ventaglio, a runtime-configurable sparse execution unit coupled with RVV ISA extensions that drives sparse tensor contractions toward their roofline through indexed gather-accumulate-scatter support. Integrated into an open-source vector processing cluster and implemented in 12nm FinFET, Ventaglio accelerates sparse tensor contraction kernels by $6.9\text{--}7.4\times$ over optimized RVV baselines, with only $3.1\%$ area overhead for a cluster of tightly-L1 coupled vector processing elements. We build a performance-accurate instruction-level model of the Ventaglio extension, calibrate it against RTL implementation, and leverage it for scale-out performance analysis on a large $4\times4$ multi-cluster system. Using a DuoGPT-pruned LLaMA-3-8B model with practical $40\text{--}60\%$ dual sparsity, Ventaglio achieves $2.40\text{--}5.25\times$ and $2.06\text{--}3.16\times$ speedup over dense baselines during prefill and autoregressive decoding, respectively.
