Search papers, labs, and topics across Lattice.
This paper introduces GRACE, a generative recommender system designed for real-time ad retrieval that effectively addresses eligibility and compute challenges. By implementing Generative Target Matching (GTM), GRACE enhances ad-level target matching rates from 23.55% to 40.42% through advanced filtering techniques, while also achieving significant reductions in latency and GPU costs. The system optimizes encoder-decoder Transformers, achieving up to 68 times faster cross-attention latency compared to existing methods, thus enabling efficient generation of thousands of ads per request.
GRACE accelerates real-time ad retrieval by improving eligibility matching rates and drastically reducing latency, making it feasible to generate thousands of ads on demand.
Productionizing generative recommenders for high-volume, real-time ads retrieval creates two serving challenges: eligibility, ensuring that each generated ad is eligible for the request under the advertiser's audience targeting rules, and compute, which requires meeting strict latency and GPU cost requirements while remaining capable of generating thousands of ads per request with wide-beam decoding. This paper presents GRACE, a serving system for ads generative retrieval that addresses both challenges. For eligibility, GRACE introduces Generative Target Matching (GTM), which extends catalog-valid constrained decoding with personalized filtering over Semantic ID (SID) prefixes using bitmask and Bloom filter matchers derived from targeting rules. SID-level GTM improves final ad-level target matching pass rate from 23.55% to 40.42% over constrained decoding alone. For compute-cost and latency, GRACE targets encoder-decoder Transformers, which are more lightweight than LLMs. It redesigns the decoder around the wide-beam, short-sequence regime, covering attention kernels, KV cache, and beam search optimizations. On NVIDIA GH200, compared with the faster of FlashAttention-2 and FlashAttention-3 baselines, GRACE improves cross-attention latency by 68.0 times and self-attention latency by 23.4-25.8 times across decode steps. Together, these changes reduce decoder latency by 11.1 times, keeping ads generative retrieval within latency and compute requirements.
