Search papers, labs, and topics across Lattice.
This paper introduces IMFuse, an instance-aware multi-layer fusion strategy that enhances sequential recommendation systems by leveraging semantic representations from multiple layers of Large Language Models (LLMs). The authors identify that relying solely on final-layer hidden states leads to dimensional collapse and loss of valuable semantic information, while intermediate layers retain complementary knowledge. Through extensive experimentation, IMFuse shows a significant average improvement of 6.72% over state-of-the-art methods, demonstrating its effectiveness in generating personalized recommendations with minimal additional computational cost.
Relying on just the final layer of LLMs can lead to a 6.72% drop in recommendation performance鈥擨MFuse captures the full spectrum of semantic knowledge across layers for superior results.
Recent advancements in Large Language Models (LLMs) have significantly enhanced sequential recommendation by encoding rich item textual information into semantic representations. However, existing methods typically rely on the final-layer hidden states of LLMs, overlooking potentially useful semantic signals encoded in other layers. Through empirical analysis, we reveal the limitations of this practice: final-layer representations often suffer from dimensional collapse, whereas intermediate layers preserve complementary, coarse-to-fine semantic knowledge. Furthermore, we observe that different items exhibit heterogeneous layer-wise representation evolution, making a uniform layer selection sub-optimal. To bridge this gap, we propose IMFuse, an instance-aware multi-layer fusion strategy designed for LLM-enhanced recommendation. Instead of relying on a single layer, IMFuse adaptively aggregates multi-layer semantic information by learning global dimension-wise layer preferences to capture general semantic contributions. To address item-level heterogeneity, IMFuse introduces an instance-aware expert modulation mechanism that dynamically adjusts these global preferences, generating personalized, item-specific semantic representations. Extensive experiments across four real-world datasets demonstrate the effectiveness of IMFuse. It consistently outperforms state-of-the-art baselines with an average relative improvement of 6.72%, while introducing limited parameter and computational overhead.
