Search papers, labs, and topics across Lattice.
The paper introduces ILRec, a preference fine-tuning framework for LLM-based recommendation that leverages self-hard negative signals extracted from intermediate layers to improve preference learning. ILRec identifies self-hard negative tokens from intermediate layers as fine-grained negative supervision, dynamically reflecting the model's preference learning process. The framework employs a two-stage approach involving cross-layer preference optimization and distillation, along with a collaborative filtering model for token-level reward assignment, demonstrating improved performance on three datasets.
LLMs learn to recommend better by looking inside themselves, using intermediate layer activations to generate harder negatives on the fly.
Large language models (LLMs) have shown great promise in recommender systems, where supervised fine-tuning (SFT) is commonly used for adaptation. Subsequent studies further introduce preference learning to incorporate negative samples into the training process. However, existing methods rely on sequence-level, offline-generated negatives, making them less discriminative and informative when adapting LLMs to recommendation tasks with large negative item spaces. To address these challenges, we propose ILRec, a novel preference fine-tuning framework for LLM-based recommendation, leveraging self-hard negative signals extracted from intermediate layers to improve preference learning. Specifically, we identify self-hard negative tokens from intermediate layers as fine-grained negative supervision that dynamically reflects the model's preference learning process. To effectively integrate these signals into training, we design a two-stage framework comprising cross-layer preference optimization and cross-layer preference distillation, enabling the model to jointly discriminate informative negatives and enhance the quality of negative signals from intermediate layers. In addition, we introduce a lightweight collaborative filtering model to assign token-level rewards for negative signals, mitigating the risk of over-penalizing false negatives. Extensive experiments on three datasets demonstrate ILRec's effectiveness in enhancing the performance of LLM-based recommender systems.
