用于8192标记双语文本嵌入的多任务对比学习
计算与语言
2024-02-28 v1 人工智能
信息检索
摘要
我们介绍了一个新的一套最先进的双语文本嵌入模型，旨在支持英文和另一种目标语言。这些模型能够处理最高达8192个标记的长文本输入，使其在文本检索、聚类和语义文本相似性(STS)计算等自然语言处理任务中具有高度的灵活性。通过关注双语模型并引入独特的多任务学习目标，我们显著提高了模型在STS任务上的性能，这超越了现有多语言模型在目标语言理解和跨语言评估任务中的能力。此外，我们的双语模型更高效，需要更少的参数和内存，因为其词表更小。 Furthermore, we have expanded the Massive Text Embedding Benchmark (MTEB) to include benchmarks for German and Spanish embedding models. This integration aims to stimulate further research and advancement in text embedding technologies for these languages.
引用
@article{arxiv.2402.17016,
title = {Multi-Task Contrastive Learning for 8192-Token Bilingual Text Embeddings},
author = {Isabelle Mohr and Markus Krimmel and Saba Sturua and Mohammad Kalim Akram and Andreas Koukounas and Michael Günther and Georgios Mastrapas and Vinit Ravishankar and Joan Fontanals Martínez and Feng Wang and Qi Liu and Ziniu Yu and Jie Fu and Saahil Ognawala and Susana Guzman and Bo Wang and Maximilian Werk and Nan Wang and Han Xiao},
journal= {arXiv preprint arXiv:2402.17016},
year = {2024}
}