Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces SISA-Rec, a transformer-based sequential recommendation framework that integrates semantic context into user preference modeling to address limitations in traditional systems that rely heavily on item identifiers. By combining item ID embeddings with BERT-based text embeddings through a gated fusion module and enhancing the self-attention mechanism with semantic similarity, SISA-Rec significantly improves recommendation accuracy in sparse and cold-start scenarios. Experimental results on the Amazon Beauty and Amazon Toys & Games datasets show that SISA-Rec outperforms state-of-the-art models, achieving notable gains in hit rate and NDCG metrics, particularly for users with limited historical interactions.
Integrating semantic context into recommendation systems can lead to a remarkable 23.1% improvement in hit rate for cold-start users.
Recommendation systems help users recommend relevant items from a large collection of choices. Present work on transformer-based sequential recommendation learns user preferences from interaction logs, but it mostly focuses on item identifiers and doesn't fully use the semantic meaning of items. This limitation becomes a major challenge in sparse and cold-start scenarios where historical interaction data is limited. To solve this problem, we introduce SISA-Rec (Semantically Integrated Sequential Recommendation), a transformer-based framework that embeds semantic context directly into sequential modeling. Our approach fuses item ID embeddings with BERT-based text embeddings via a gated fusion module, injects semantic similarity into the self-attention mechanism, and leverages an attention-based aggregation module to construct comprehensive user representations. Finally, a joint learning objective which combines Bayesian Personalized Ranking (BPR) and contrastive alignment loss, aligns the underlying behavioral and semantic spaces. Experiments were conducted on the two highly sparse Amazon Beauty and Amazon Toys \& Games datasets, both having 99.93\% sparsity. The results show that SISA-Rec outperforms state-of-the-art baseline models across all evaluation metrics. Compared with the BERT4Rec \cite{petrov2022systematic}, SISA-Rec improves HR@10 by 16.6\% and NDCG@10 by 10.3\% on Amazon Beauty, and HR@10 by 23.1\% and NDCG@10 by 17.9\% on Amazon Toys \& Games. Cold-start analysis further shows that the proposed model achieves the largest improvements for users with limited interaction historical records. This showcases the value of semantic information when user behavior data is scarce. Overall, the results demonstrate that integrating semantic information into the attention mechanism leads to more accurate and reliable recommendations.