Search papers, labs, and topics across Lattice.
This paper introduces PromptEmbedder, a dual-LLM framework for efficient text embedding where a Prompting LLM generates soft prompts for a frozen Embedding LLM. By decoupling embedding knowledge from the backbone, PromptEmbedder achieves comparable performance to LoRA finetuning on MTEB while reducing GPU memory by 40% and accelerating training by 3.7x. The key is a differentiable generation process with continuous relaxation, enabling full gradient flow during contrastive training while only requiring retraining a lightweight linear alignment matrix for new architectures.
Forget retraining LLMs from scratch for new architectures: PromptEmbedder lets you adapt to new backbones by only retraining a lightweight linear alignment matrix, slashing memory by 40% and speeding up training by 3.7x.
Large Language Models (LLMs) have demonstrated remarkable efficacy in text embedding, yet current adaptation methods like LoRA face significant bottlenecks in computational efficiency and cross-architecture transferability. Whenever a new backbone emerges, existing approaches require costly retraining from scratch. To address this, we propose PromptEmbedder, a novel dual-LLM framework that decouples embedding knowledge from specific backbone weights. PromptEmbedder utilizes a Prompting LLM to generate instruction-aware soft prompts for a frozen Embedding LLM via a differentiable generation process with continuous relaxation, ensuring full gradient flow during contrastive training. By localizing task-specific knowledge within the Prompting LLM, adapting to new architectures requires only retraining a lightweight linear alignment matrix. Evaluations on the MTEB benchmark show that PromptEmbedder achieves comparable performance with LoRA finetuning while reducing GPU memory by 40% and accelerating training by 3.7x. Our approach establishes a scalable, architecture-agnostic paradigm for efficient LLM-based representation learning.
