Search papers, labs, and topics across Lattice.
The paper introduces FlashEvaluator, a novel evaluator designed to address limitations in the Generator-Evaluator (G-E) framework by enabling cross-sequence token information sharing and processing all sequences in a single forward pass. This approach achieves sublinear computational complexity, improving efficiency and enabling direct inter-sequence comparisons for enhanced selection accuracy. Empirical results across recommendation and NLP tasks demonstrate FlashEvaluator's superiority over traditional methods, with successful online deployment in Kuaishou's recommender system leading to significant revenue gains.
FlashEvaluator slashes the computational cost of evaluating multiple sequences in Generator-Evaluator frameworks while boosting accuracy by enabling direct cross-sequence comparisons.
The Generator-Evaluator (G-E) framework, i.e., evaluating K sequences from a generator and selecting the top-ranked one according to evaluator scores, is a foundational paradigm in tasks such as Recommender Systems (RecSys) and Natural Language Processing (NLP). Traditional evaluators process sequences independently, suffering from two major limitations: (1) lack of explicit cross-sequence comparison, leading to suboptimal accuracy; (2) poor parallelization with linear complexity of O(K), resulting in inefficient resource utilization and negative impact on both throughput and latency. To address these challenges, we propose FlashEvaluator, which enables cross-sequence token information sharing and processes all sequences in a single forward pass. This yields sublinear computational complexity that improves the system's efficiency and supports direct inter-sequence comparisons that improve selection accuracy. The paper also provides theoretical proofs and extensive experiments on recommendation and NLP tasks, demonstrating clear advantages over conventional methods. Notably, FlashEvaluator has been deployed in online recommender system of Kuaishou, delivering substantial and sustained revenue gains in practice.
