Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Long-Term Embeddings (LTE), a method to incorporate stable, long-term user preferences into transformer-based sequential recommenders to combat recency bias. LTE addresses the point-in-time consistency problem by constraining embeddings to a fixed semantic basis of content-based item representations, ensuring compatibility across model versions. Online A/B tests on Zalando's platform demonstrate that integrating LTE as a contextual prefix token with a lagged window significantly improves user engagement and financial metrics.
Forget retraining: this method lets you hot-swap recommendation models without breaking personalization, by anchoring user embeddings to a fixed semantic space.
Modern transformer-based sequential recommenders excel at capturing short-term intent but often suffer from recency bias, overlooking stable long-term preferences. While extending sequence lengths is an intuitive fix, it is computationally inefficient, and recent interactions tend to dominate the model's attention. We propose Long-Term Embeddings (LTE) as a high-inertia contextual anchor to bridge this gap. We address a critical production challenge: the point-in-time consistency problem caused by infrastructure constraints, as feature stores typically host only a single"live"version of features. This leads to an offline-online mismatch during model deployments and rollbacks, as models are forced to process evolved representations they never saw during training. To resolve this, we introduce an LTE framework that constrains embeddings to a fixed semantic basis of content-based item representations, ensuring cross-version compatibility. Furthermore, we investigate integration strategies for causal language modeling, considering the data leakage issue that occurs when the LTE and the transformer's short-term sequence share a temporal horizon. We evaluate two representations: a heuristic average and an asymmetric autoencoder with a fixed decoder grounded in the semantic basis to enable behavioral fine-tuning while maintaining stability. Online A/B tests on Zalando demonstrate that integrating LTE as a contextual prefix token using a lagged window yields significant uplifts in both user engagement and financial metrics.