Search papers, labs, and topics across Lattice.
This paper introduces Tlow, a flow-based item tokenizer that encodes semantic embeddings into token IDs, addressing the inefficiencies of traditional recommendation models like RQ-VAE and optimized product quantization. By transforming embeddings into a latent space with a unified standard normal distribution, Tlow achieves both dimensional independence and distributional simplicity, which enhances the clarity of token IDs. Experimental results reveal that Tlow significantly boosts recommendation performance, particularly in cross-domain and multi-modal contexts, with a notable increase in user click-through rates on WeChat.
Tlow's innovative approach to item tokenization improves recommendation performance by over 10% in user engagement, even for cold-start items.
Item tokenizer encodes semantic embeddings into token IDs to replace the randomly assigned item IDs used in traditional recommendation models, fundamentally addressing the problems of excessive parameters and cold starts. However, the most common tokenizer, RQ-VAE, suffers from low decoding efficiency due to the inherent dependencies among its codebooks. Meanwhile, efficient independent tokenizers such as optimized product quantization (OPQ) still struggle with dimensional correlations and distribution complexity of semantic embeddings. In this work, we propose a f\underline{low}-based item \underline{T}okenizer (Tlow) to transform raw semantic embeddings into a latent space where embeddings conform to a unified standard normal distribution, achieving dual advantages of dimensional independence and distributional simplicity. Independent tokenization performed on these latent embeddings yields semantically clear token IDs. Additionally, we introduce a novel codebook guidance to align the codebook space with the token embedding space, further aiding the learning of more semantically distinct token embeddings. Offline experiments on four public datasets demonstrate that Tlow's tokenization and codebook guidance significantly improve recommendation performance. The improvement on cross-domain and multi-modal recommendations also proves the effectiveness of item tokenization in a simplified embedding space. Online experiments for a multi-modal retrieval task on China's largest social media platform WeChat validate Tlow's powerful distribution transformation capability. The retrieval model based on token IDs improves user CTR by 10.32\% globally and by 11.64\% for new items. Our codes are available at https://github.com/wjjln/Tlow.
