Search papers, labs, and topics across Lattice.
This paper investigates the compression of LLM-derived representations for recommendation systems and identifies a "Mid-layer Representation Advantage" (MRA) where middle layers outperform final layers. To address this, they propose Modular Representation Compression (MARC), which explicitly controls LLM modularity by introducing compression and task adaptation modules while decoupling tasks with information constraints. Online A/B testing demonstrates a 2.82% eCPM lift in a commercial search advertising setting, validating the effectiveness of MARC in producing efficient representations.
LLMs' final layers might be holding your recommendation system back: representations from middle layers actually perform *better*, and a modular compression approach can unlock significant gains.
Recently, large language models (LLMs) have advanced recommendation systems (RSs), and recent works have begun to explore how to integrate LLMs into industrial RSs. While most approaches deploy LLMs offline to generate and pre-cache augmented representations for RSs, high-dimensional representations from LLMs introduce substantial storage and computational costs. Thus, it is crucial to compress LLM representations effectively. However, we identify a counterintuitive phenomenon during representation compression: Mid-layer Representation Advantage (MRA), where representations from middle layers of LLMs outperform those from final layers in recommendation tasks. This degraded final layer renders existing compression methods, which typically compress on the final layer, suboptimal. We interpret this based on modularity theory that LLMs develop spontaneous internal functional modularity and force the final layer to specialize in the proxy training task. Thus, we propose \underline{M}odul\underline{a}r \underline{R}epresentation \underline{C}ompression (MARC) to explicitly control the modularity of LLMs. First, Modular Adjustment explicitly introduces compression and task adaptation modules, enabling the LLM to operate strictly as a representation-learning module. Next, to ground each module to its specific task, Modular Task Decoupling uses information constraints and different network structures to decouple tasks. Extensive experiments validate that MARC addresses MRA and produces efficient representations. Notably, MARC achieved a 2.82% eCPM lift in an online A/B test within a large-scale commercial search advertising scenario.
