Search papers, labs, and topics across Lattice.
This paper introduces CAGR, a Cross-Accelerator Graph Optimization Framework designed to enhance the efficiency of recommender system inference across diverse hardware accelerators. By employing a hardware-aware graph rewriting engine, a reinforcement learning-based transformation policy, and a heterogeneous pipeline architecture, CAGR achieves significant performance portability and optimization. Experimental results indicate that CAGR provides a 1.8 to 3.2 times speedup over baseline implementations while maintaining high performance and reducing optimization time by 67% compared to traditional methods.
Achieving up to 3.2x speedup in recommender system inference across multiple hardware platforms without manual tuning could revolutionize deployment strategies in heterogeneous environments.
Recommender systems have become ubiquitous in modern online services, yet their deployment across diverse hardware accelerators remains challenging due to significant performance variations. Contemporary deep learning recommendation models (DLRMs), such as DeepFM and NGCF, exhibit substantial inference latency differences when executed on NVIDIA GPUs, AMD GPUs, and Google TPUs, primarily due to architectural disparities and vendor-specific optimization strategies. Existing graph optimization frameworks are typically designed for specific hardware backends, lacking the flexibility to generate portable high-performance implementations across heterogeneous accelerators. This paper presents CAGR (Cross-Accelerator Graph Rewriting), a novel framework that achieves performance-portable inference optimization for recommendation models through three key innovations: 1) hardware-aware graph rewriting engine that dynamically selects optimal operator implementations by analyzing compute-to-memory bandwidth ratios and operator density characteristics; 2) reinforcement learning-based transformation policy that learns cross-platform optimization strategies without exhaustive search; and 3) heterogeneous pipeline architecture enabling “optimize once, deploy across supported backends” semantics. We implement CAGR with support for multiple kernel backends, including Triton, cuBLAS, MIOpen, and oneDNN, and demonstrate its effectiveness on the Avazu CTR prediction dataset. Experimental results show that CAGR achieves 1.8- $3.2\times $ speedup over baseline implementations across NVIDIA V100, AMD MI100, and Google TPU v3 platforms, while reducing optimization time by 67% compared to platform-specific auto-tuning approaches. Furthermore, CAGR maintains 92-96% of reference optimized performance with zero manual intervention, demonstrating practical viability for production recommendation systems requiring multi-vendor deployment.
CMU ML
Microsoft Research