Search papers, labs, and topics across Lattice.
FLAME addresses the challenge of capturing diverse user behaviors in sequential recommendation by condensing an ensemble of diverse representations into a single network. It uses a modular ensemble approach with a frozen, pretrained network and a learnable network to simulate exponential diversity during training, dynamically combining sub-modules of each network. This allows the learnable network to learn diverse representations guided by the frozen network, achieving ensemble-level performance at inference with no additional overhead, and significantly improving convergence speed and recommendation accuracy.
Achieve ensemble-level sequential recommendation performance with a single network at inference time by distilling diversity from a modular ensemble during training.
Sequential recommendation requires capturing diverse user behaviors, which a single network often fails to capture. While ensemble methods mitigate this by leveraging multiple networks, training them all from scratch leads to high computational cost and instability from noisy mutual supervision. We propose {\bf F}rozen and {\bf L}earnable networks with {\bf A}ligned {\bf M}odular {\bf E}nsemble ({\bf FLAME}), a novel framework that condenses ensemble-level diversity into a single network for efficient sequential recommendation. During training, FLAME simulates exponential diversity using only two networks via {\it modular ensemble}. By decomposing each network into sub-modules (e.g., layers or blocks) and dynamically combining them, FLAME generates a rich space of diverse representation patterns. To stabilize this process, we pretrain and freeze one network to serve as a semantic anchor and employ {\it guided mutual learning}. This aligns the diverse representations into the space of the remaining learnable network, ensuring robust optimization. Consequently, at inference, FLAME utilizes only the learnable network, achieving ensemble-level performance with zero overhead compared to a single network. Experiments on six datasets show that FLAME outperforms state-of-the-art baselines, achieving up to 7.69$\times$ faster convergence and 9.70\% improvement in NDCG@20. We provide the source code of FLAME at https://github.com/woo-joo/FLAME_SIGIR26.
