Search papers, labs, and topics across Lattice.
This paper introduces SCoRD, a continual knowledge distillation framework designed to enhance LLM-based recommendation systems by enabling efficient co-adaptation of retrievers and rerankers in response to non-stationary data streams. By leveraging a semantic reasoning assistant, SCoRD selectively distills knowledge from the LLM reranker to the retriever, focusing on low-confidence sequences while minimizing the need for repeated LLM inference. Experimental results demonstrate that SCoRD significantly improves retrieval quality and adaptability, making it a cost-effective solution for evolving user interests in recommendation systems.
SCoRD transforms LLM-based recommendation systems by enabling retrievers to adapt intelligently to user intent without the heavy cost of frequent LLM updates.
Recommendation systems increasingly adopt a two-stage pipeline, where an ID-based retriever retrieves candidates and an LLM-based reranker refines their rankings. To improve retrieval quality, reranker-to-retriever distillation is commonly used to transfer the reranker's knowledge to the retriever. For practical deployment, however, this pipeline must continually adapt to evolving interests and incoming interactions. A naive solution is to repeatedly update the LLM reranker and distill its latest knowledge, but this incurs prohibitive costs. Updating the retriever alone is cheaper, but its limited capacity makes adaptation from sparse data difficult. We propose SCoRD, a continual knowledge distillation framework for LLM-based reranking pipelines under a non-stationary data stream. SCoRD introduces a semantic reasoning assistant that distills the LLM's ability to infer underlying user intents into reusable intent-level guidance. It selectively distills reranker knowledge to the retriever on low-confidence sequences, guides retriever-only updates without repeated LLM inference, and feeds retriever-derived representations and intent-drift signals back to the reranker. Experiments on real-world datasets show that SCoRD enables effective and efficient retriever-reranker co-adaptation.
