Large Language Models (LLMs) have demonstrated remarkable efficacy in text embedding, yet current adaptation methods like LoRA face significant bottlenecks in computational efficiency and cross-architecture transferability. Whenever a new backbone emerges, existing approaches require costly retraining from scratch. To address this, we propose PromptEmbedder, a novel dual-LLM framework that decouples embedding knowledge from specific backbone weights. PromptEmbedder utilizes a Prompting LLM to generate instruction-aware soft prompts for a frozen Embedding LLM via a differentiable generation process with continuous relaxation, ensuring full gradient flow during contrastive training. By localizing task-specific knowledge within the Prompting LLM, adapting to new architectures requires only retraining a lightweight linear alignment matrix. Evaluations on the MTEB benchmark show that PromptEmbedder achieves comparable performance with LoRA finetuning while reducing GPU memory by 40% and accelerating training by 3.7x. Our approach establishes a scalable, architecture-agnostic paradigm for efficient LLM-based representation learning.
@article{arxiv.2605.28066,
title = {PromptEmbedder:: Efficient and Transferable Text Embedding via Dual-LLM Soft Prompting},
author = {Yu-Che Tsai and Kuan-Yu Chen and Yuan-Hao Chen and Yu-Han Chang and Ching-Yu Tsai and Yu-Hsiang Chuang and Shou-De Lin},
journal= {arXiv preprint arXiv:2605.28066},
year = {2026}
}