Search papers, labs, and topics across Lattice.
The paper introduces TrieRec, a trie-aware generative recommendation method that leverages the hierarchical structure of item tokens in generative recommendation systems. It addresses the limitation of standard Transformers that flatten item tokens by incorporating structural inductive biases through two novel positional encodings. TrieRec demonstrates significant improvements in next-item prediction accuracy, achieving an average of 8.83% improvement across four real-world datasets when integrated with three representative generative recommendation backbones.
Generative recommendation gets a boost: TrieRec leverages the inherent hierarchical structure of item tokens to significantly improve next-item prediction accuracy.
Generative recommendation (GR) aligns with advances in generative AI by casting next-item prediction as token-level generation rather than score-based ranking. Most GR methods adopt a two-stage pipeline: (i) \textit{item tokenization}, which maps each item to a sequence of discrete, hierarchically organized tokens; and (ii) \textit{autoregressive generation}, which predicts the next item's tokens conditioned on the tokens of user's interaction history. Although hierarchical tokenization induces a prefix tree (trie) over items, standard autoregressive modeling with conventional Transformers often flattens item tokens into a linear stream and overlooks the underlying topology. To address this, we propose TrieRec, a trie-aware generative recommendation method that augments Transformers with structural inductive biases via two positional encodings. First, a \textit{trie-aware absolute positional encoding} aggregates a token's (node's) local structural context (\eg depth, ancestors, and descendants) into the token representation. Second, a \textit{topology-aware relative positional encoding} injects pairwise structural relations into self-attention to capture topology-induced semantic relatedness. TrieRec is also model-agnostic, efficient, and hyperparameter-free. In our experiments, we implement TrieRec within three representative GR backbones, achieving notably improvements of 8.83\% on average across four real-world datasets.
