Search papers, labs, and topics across Lattice.
This paper introduces SSR-GRPO, a novel approach that enhances reinforcement learning for dense retrieval in e-commerce by integrating supervision and semantic identifiers to improve relevance assessments. The method addresses the limitations of existing retrieval models, such as noisy candidate selection and biased relevance scoring, by employing a dual-perspective framework and mining hard negative samples. Experimental results demonstrate SSR-GRPO's superior performance in both offline and online settings, leading to its successful deployment on a large-scale e-commerce platform.
SSR-GRPO significantly reduces noise in retrieval systems, enhancing relevance assessments and enabling more accurate e-commerce search results.
Embedding-based retrieval (EBR) is pivotal in e-commerce search but often struggles with complex semantics. While recent methods often fine-tune large language models (LLMs) for representation learning, they typically lack robust mechanisms for handling complex and implicit semantics. While Retrieval-GRPO (R-GRPO) recently introduced reinforcement learning to dense retrieval, it suffers from noisy top-K candidates due to limited batch sampling and biased relevance assessments caused by using similarly trained LLMs as reward models. To tackle these issues, we propose Supervised Retrieval-GRPO with Semantic Identifiers (SSR-GRPO). Specifically, our method first proposes a dual-perspective framework for relevance assessment. It leverages both Semantic Identifiers (SIDs) produced by quantization learning and dense representation vectors to generate more unbiased relevance scores. Furthermore, leveraging the hierarchical similarity relationships of the generated SIDs, we mine a set of hard negative samples that serve two purposes: (1) to design a masking function integrated into R-GRPO, effectively filtering intra-group noisy samples; and (2) to construct a Retrieval-DPO task composed of positive and negative sample pairs, enabling the model to capture fine-grained semantic distinctions from a pair-wise perspective. By integrating these optimization strategies, we propose SSR-GRPO. Extensive offline and online experiments validate SSR-GRPO's effectiveness, and it has been deployed on a large-scale e-commerce platform.
