English
Related papers

Related papers: Multilingual-To-Multimodal (M2M): Unlocking New La…

200 papers

Recently, numbers of works shows that the performance of neural machine translation (NMT) can be improved to a certain extent with using visual information. However, most of these conclusions are drawn from the analysis of experimental…

Computer Vision and Pattern Recognition · Computer Science 2022-09-07 ZhenHao Tang , XiaoBing Zhang , Zi Long , XiangHua Fu

Pre-training decoder-only language models relies on vast amounts of high-quality data, yet the availability of such data is increasingly reaching its limits. While metadata is commonly used to create and curate these datasets, its potential…

Computation and Language · Computer Science 2025-12-09 Sebastian Sztwiertnia , Felix Friedrich , Kristian Kersting , Patrick Schramowski , Björn Deiseroth

We introduce Gemini Embedding 2, a native multimodal embedding model that allows embedding video, audio, image, and text modalities in a unified representation space. We leverage the multimodal capabilities of Gemini to produce embeddings…

Current Text-to-Speech models pose a multilingual challenge, where most of the models traditionally focus on English and European languages, thereby hurting the potential to provide access to information to many more people. To address this…

Audio and Speech Processing · Electrical Eng. & Systems 2025-08-21 Jaskaran Singh , Amartya Roy Chowdhury , Raghav Prabhakar , Varshul C. W

The evolution from Large Language Models (LLMs) to Multimodal Large Language Models (MLLMs) has spurred research into extending In-Context Learning (ICL) to its multimodal counterpart. Existing such studies have primarily concentrated on…

Machine Learning · Computer Science 2024-07-23 Yuchen Zeng , Wonjun Kang , Yicong Chen , Hyung Il Koo , Kangwook Lee

We introduce llama-embed-nemotron-8b, an open-weights text embedding model that achieves state-of-the-art performance on the Multilingual Massive Text Embedding Benchmark (MMTEB) leaderboard as of October 21, 2025. While recent models show…

Computation and Language · Computer Science 2025-11-11 Yauhen Babakhin , Radek Osmulski , Ronay Ak , Gabriel Moreira , Mengyao Xu , Benedikt Schifferer , Bo Liu , Even Oldridge

This work presents a generalizable framework to transfer relative depth to metric depth. Current monocular depth estimation methods are mainly divided into metric depth estimation (MMDE) and relative depth estimation (MRDE). MMDEs estimate…

Computer Vision and Pattern Recognition · Computer Science 2026-03-20 Beilei Cui , Yiming Huang , Long Bai , Hongliang Ren

Text-rich images, where text serves as the central visual element guiding the overall understanding, are prevalent in real-world applications, such as presentation slides, scanned documents, and webpage snapshots. Tasks involving multiple…

Computer Vision and Pattern Recognition · Computer Science 2025-06-09 Mengzhao Jia , Wenhao Yu , Kaixin Ma , Tianqing Fang , Zhihan Zhang , Siru Ouyang , Hongming Zhang , Dong Yu , Meng Jiang

Recent Multimodal Large Language Models (MLLMs) have significantly advanced e-commerce product understanding. However, they still face three challenges: (i) the modality imbalance induced by modality mixed training; (ii) underutilization of…

Computer Vision and Pattern Recognition · Computer Science 2026-03-25 Zhanheng Nie , Chenghan Fu , Daoze Zhang , Junxian Wu , Wanxian Guan , Pengjie Wang , Jian Xu , Bo Zheng

Multimodal few-shot learning is challenging due to the large domain gap between vision and language modalities. Existing methods are trying to communicate visual concepts as prompts to frozen language models, but rely on hand-engineered…

Computer Vision and Pattern Recognition · Computer Science 2023-03-01 Ivona Najdenkoska , Xiantong Zhen , Marcel Worring

Modern software systems are increasingly shifting from monolithic architectures to microservices to enhance scalability, maintainability, and deployment flexibility. Existing microservice extraction methods typically rely on hard…

Software Engineering · Computer Science 2025-08-12 Morteza Ziabakhsh , Kiyan Rezaee , Sadegh Eskandari , Seyed Amir Hossein Tabatabaei , Mohammad M. Ghassemi

Multimodal retrieval systems are expected to operate in a semantic space, agnostic to the language or cultural origin of the query. In practice, however, retrieval outcomes systematically reflect perspectival biases: deviations shaped by…

Multimodal Large Language Models (MLLMs) have shown significant promise in various applications, leading to broad interest from researchers and practitioners alike. However, a comprehensive evaluation of their long-context capabilities…

Machine Learning · Computer Science 2025-02-12 Hengyi Wang , Haizhou Shi , Shiwei Tan , Weiyi Qin , Wenyuan Wang , Tunyu Zhang , Akshay Nambi , Tanuja Ganu , Hao Wang

Research on Multi-modal Large Language Models (MLLMs) towards the multi-image cross-modal instruction has received increasing attention and made significant progress, particularly in scenarios involving closely resembling images (e.g.,…

Computer Vision and Pattern Recognition · Computer Science 2024-08-26 Tao Wu , Mengze Li , Jingyuan Chen , Wei Ji , Wang Lin , Jinyang Gao , Kun Kuang , Zhou Zhao , Fei Wu

This technical report presents the training methodology and evaluation results of the open-source multilingual E5 text embedding models, released in mid-2023. Three embedding models of different sizes (small / base / large) are provided,…

Computation and Language · Computer Science 2024-02-09 Liang Wang , Nan Yang , Xiaolong Huang , Linjun Yang , Rangan Majumder , Furu Wei

We present a Multi-Modal Recipe for Advancing Adaptation-based Pre-training towards effective and efficient zero-shot video-text retrieval, dubbed M2-RAAP. Upon popular image-text models like CLIP, most current adaptation-based video-text…

Computer Vision and Pattern Recognition · Computer Science 2024-02-01 Xingning Dong , Zipeng Feng , Chunluan Zhou , Xuzheng Yu , Ming Yang , Qingpei Guo

We propose MindVL, a multimodal large language model (MLLMs) trained on Ascend NPUs. The training of state-of-the-art MLLMs is often confined to a limited set of hardware platforms and relies heavily on massive, undisclosed data recipes,…

Computer Vision and Pattern Recognition · Computer Science 2025-10-01 Feilong Chen , Yijiang Liu , Yi Huang , Hao Wang , Miren Tian , Ya-Qi Yu , Minghui Liao , Jihao Wu

The development of high-quality text embeddings is increasingly drifting toward an exclusionary future, defined by three critical barriers: prohibitive computational costs, a narrow linguistic focus that neglects most of the world's…

Computation and Language · Computer Science 2026-05-15 Ziyin Zhang , Zihan Liao , Hang Yu , Peng Di , Rui Wang

Recent multilingual pretrained language models (mPLMs) often avoid using language embeddings -- learnable vectors assigned to individual languages. However, this places a significant burden on token representations to encode all…

Computation and Language · Computer Science 2025-05-23 Yihong Liu , Haotian Ye , Chunlan Ma , Mingyang Wang , Hinrich Schütze

Multilingual semantic parsing is a cost-effective method that allows a single model to understand different languages. However, researchers face a great imbalance of availability of training data, with English being resource rich, and other…

Computation and Language · Computer Science 2021-06-15 Menglin Xia , Emilio Monti