Search papers, labs, and topics across Lattice.
This paper investigates why LLM-enhanced recommendation models suffer from optimization challenges during training, identifying large norm disparity and semantic-collaboration misalignment as key culprits. To mitigate these issues, they propose TF-LLMER, a framework that incorporates item embedding normalization and Rec-PCA, a recommendation-aware dimensionality reduction technique. Empirical results demonstrate that TF-LLMER significantly improves performance compared to existing LLM-enhanced recommendation methods by addressing optimization instability.
LLM-enhanced recommenders stumble because of representation norm disparities and semantic misalignment, but a simple normalization and PCA-inspired alignment can unlock their potential.
Large language model (LLM)-enhanced recommendation models inject LLM representations into backbone recommenders to exploit rich item text without inference-time LLM cost. However, we find that existing LLM-enhanced methods significantly hinder the optimization of backbone models, resulting in high training losses that are difficult to reduce. To address it, we establish a comprehensive theoretical analysis of local optimization curvature and identify two key causes: 1) large norm disparity and 2) semantic-collaboration misaligned angular clustering of LLM representations. Guided by these insights, we propose Training-Friendly LLM-Enhanced Recommender (TF-LLMER), a lightweight framework with two key components. First, we highlight the necessity of item embedding normalization to eliminate norm-driven instability and achieve provable control over optimization conditioning. Second, we introduce Rec-PCA, a recommendation-aware dimensionality reduction method that injects collaborative structure into the representation transformation to resolve semantic-collaboration misaligned angular clustering. It jointly optimizes semantic information retention and alignment with an item-item co-occurrence graph constructed from interaction histories. The graph captures collaborative structure, and alignment is promoted by penalizing total variation over the graph. Both theory and extensive experiments demonstrate that TF-LLMER significantly outperforms state-of-the-art methods. Our code is available at https://github.com/woriazzc/TF-LLMER.
