Search papers, labs, and topics across Lattice.
This paper introduces a novel approach to text embeddings using recurrent architectures that leverage a vertically chunked inference strategy, significantly improving computational efficiency. By fine-tuning Mamba2 models, the authors achieve competitive performance on various benchmarks while maintaining constant memory usage as input length increases, contrasting sharply with the quadratic complexity of transformer-based models. Empirical validation across Mamba2, RWKV, and xLSTM confirms that recurrent models can serve as a viable alternative for efficient embedding generation, offering a promising direction for long-sequence processing in NLP.
Recurrent models can achieve competitive text embedding performance with constant memory usage, challenging the dominance of transformers in long-sequence tasks.
Transformer-based embedding models suffer from quadratic computational and linear memory complexity, limiting their utility for long sequences. We propose recurrent architectures as an efficient alternative, introducing a vertically chunked inference strategy that enables fast embedding generation with memory usage that becomes constant in the input length once it exceeds the vertical chunk size. By fine-tuning Mamba2 models, we demonstrate their viability as general-purpose text embedders, achieving competitive performance across a range of benchmarks while maintaining a substantially smaller memory footprint compared to transformer-based counterparts. We empirically validate the applicability of our inference strategy to Mamba2, RWKV, and xLSTM models, confirming consistent runtime-memory trade-offs across architectures and establishing recurrent models as a compelling alternative to transformers for efficient embedding generation.
