Search papers, labs, and topics across Lattice.
This paper introduces a hybrid approach that integrates contrastive representation learning (CoLES) with selective State Space Models (SSMs) for user-centric modeling of transactional event sequences. By addressing the limitations of traditional encoders like RNNs and Transformers, the proposed Mamba model improves efficiency in handling long-range dependencies while enhancing user representation. Experiments show that the hybrid models not only converge 2–3 times faster than the baseline but also provide valuable insights through explainability analysis, highlighting key transaction features.
Hybridizing contrastive learning with selective state space models accelerates convergence and enhances explainability in user-centric transactional modeling.
We propose a hybrid approach for user-centric modeling of transactional event sequences that combines contrastive representation learning (CoLES) with State Space Models (SSMs). While contrastive methods yield high-quality compressed user representations, existing encoders -- RNNs and Transformers -- suffer from vanishing gradients or quadratic complexity, respectively. Mamba, a selective SSM, efficiently handles long-range dependencies but remains underexplored for personalized user analysis. We investigate two integration strategies: (1)~initializing the Mamba hidden state with a CoLES embedding, and (2)~prepending the projected CoLES embedding as a prefix token to the input sequence. Both approaches supply the model with an informative user prior from the first step. Experiments on three public datasets -- Age (multiclass age-group prediction), MBD (multi-label product acquisition), and Taobao (binary purchase prediction) -- demonstrate consistent improvements over standalone Mamba and CoLES with a linear classifier, with the hybrid models converging 2--3$\times$ faster than the plain SSM baseline. Explainability analysis via discretization-step maps and Integrated Gradients reveals selective event filtering on behavior-rich datasets and identifies the most informative transaction features.
