Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces Anisotropy-Controllable Embedding (ACE), a method to mitigate anisotropy in LLM-generated item embeddings used for sequential recommendation. ACE employs a linear autoencoder with L2 regularization to reshape the embedding distribution, balancing geometric uniformity and semantic preservation. Experiments show ACE improves Recall@20 and NDCG@20 by up to 12.4% and 11.8% respectively, compared to existing LLM-enhanced SR models.
LLM-generated embeddings, despite their semantic richness, can cripple sequential recommendation due to anisotropy, but a simple autoencoder can fix it.
Recent advances in the LLM-as-Extractor paradigm leverage large language models (LLMs) to transfer semantically rich item embeddings into sequential recommendation (SR) backbones. However, LLM-generated embeddings often suffer from strong anisotropy. Most vectors are concentrated in similar directions, resulting in a geometric imbalance that makes it difficult to adapt to collaborative signals during fine-tuning. To address this challenge, we propose Anisotropy-Controllable Embedding (ACE), which explicitly controls the anisotropy of LLM-generated embeddings. Specifically, ACE utilizes a linear autoencoder (LAE) to reshape the embedding distribution while preserving its semantic structure. In this process, the L2-regularization term mitigates the anisotropy by controlling the dispersion of embedding dimensions, while the reconstruction loss maintains semantic relationships among items. That is, ACE balances geometric uniformity and semantic embedding preservation for more stable learning. Extensive experiments demonstrate that ACE consistently outperforms existing LLM-enhanced SR models, yielding improvements of up to 12.4% and 11.8% in Recall@20 and NDCG@20, respectively.