English
Related papers

Related papers: Large Multi-modal Encoders for Recommendation

200 papers

True understanding of videos comes from a joint analysis of all its modalities: the video frames, the audio track, and any accompanying text such as closed captions. We present a way to learn a compact multimodal feature representation that…

Computer Vision and Pattern Recognition · Computer Science 2020-04-07 Vivek Sharma , Makarand Tapaswi , Rainer Stiefelhagen

Large multimodal language models have demonstrated impressive capabilities in understanding and manipulating images. However, many of these models struggle with comprehending intensive textual contents embedded within the images, primarily…

Computer Vision and Pattern Recognition · Computer Science 2024-07-30 Ruiyi Zhang , Yufan Zhou , Jian Chen , Jiuxiang Gu , Changyou Chen , Tong Sun

In recent years, multimodal large language models (MLLMs) have made significant strides by training on vast high-quality image-text datasets, enabling them to generally understand images well. However, the inherent difficulty in explicitly…

Computer Vision and Pattern Recognition · Computer Science 2024-07-08 Yuanze Lin , Yunsheng Li , Dongdong Chen , Weijian Xu , Ronald Clark , Philip Torr , Lu Yuan

Large Language Models (LLMs) have demonstrated remarkable capabilities and have been extensively deployed across various domains, including recommender systems. Prior research has employed specialized \textit{prompts} to leverage the…

Information Retrieval · Computer Science 2024-04-02 Sichun Luo , Bowei He , Haohan Zhao , Wei Shao , Yanlin Qi , Yinya Huang , Aojun Zhou , Yuxuan Yao , Zongpeng Li , Yuanzhang Xiao , Mingjie Zhan , Linqi Song

Recommender systems are tasked to infer users' evolving preferences and rank items aligned with their intents, which calls for in-depth reasoning beyond pattern-based scoring. Recent efforts start to leverage large language models (LLMs)…

Information Retrieval · Computer Science 2026-02-16 Kehan Zheng , Deyao Hong , Qian Li , Jun Zhang , Huan Yu , Jie Jiang , Hongning Wang

Conventional recommendation systems succeed in identifying relevant content but often fail to provide users with surprising or novel items. Multimodal Large Language Models (MLLMs) possess the world knowledge and multimodal understanding…

Information Retrieval · Computer Science 2025-09-23 Haoting Wang , Jianling Wang , Hao Li , Fangjun Yi , Mengyu Fu , Youwei Zhang , Yifan Liu , Liang Liu , Minmin Chen , Ed H. Chi , Lichan Hong , Haokai Lu

Large Language Models (LLMs) pose a new paradigm of modeling and computation for information tasks. Recommendation systems are a critical application domain poised to benefit significantly from the sequence modeling capabilities and world…

Multimodal learning has become a pivotal approach in developing robust learning models with applications spanning multimedia, robotics, large language models, and healthcare. The efficiency of multimodal systems is a critical concern, given…

Machine Learning · Computer Science 2025-03-04 Zhe Gao , Jian Huang , Ting Li , Xueqin Wang

Large Language Models (LLMs) have demonstrated great potential for assisting developers in their daily development. However, most research focuses on generating correct code, how to use LLMs to generate personalized code has seldom been…

Computation and Language · Computer Science 2024-09-27 Zhenlong Dai , Chang Yao , WenKang Han , Ying Yuan , Zhipeng Gao , Jingyuan Chen

The Multimodal Large Language Models (MLLMs) have activated the capabilitiesof Large Language Models (LLMs) in solving visual-language tasks by integratingvisual information. The prevailing approach in existing MLLMs involvesemploying an…

Computer Vision and Pattern Recognition · Computer Science 2024-10-31 Tianxiang Wu , Minxin Nie , Ziqiang Cao

Large Language Models (LLMs) have emerged as powerful tools in the field of Natural Language Processing (NLP) and have recently gained significant attention in the domain of Recommendation Systems (RS). These models, trained on massive…

Information Retrieval · Computer Science 2024-06-19 Likang Wu , Zhi Zheng , Zhaopeng Qiu , Hao Wang , Hongchao Gu , Tingjia Shen , Chuan Qin , Chen Zhu , Hengshu Zhu , Qi Liu , Hui Xiong , Enhong Chen

Learning high-quality text representations is fundamental to a wide range of NLP tasks. While encoder pretraining has traditionally relied on Masked Language Modeling (MLM), recent evidence suggests that decoder models pretrained with…

Recommender systems have become integral to our digital experiences, from online shopping to streaming platforms. Still, the rationale behind their suggestions often remains opaque to users. While some systems employ a graph-based approach,…

The integration of Large Language Models into recommendation frameworks presents key advantages for personalization and adaptability of experiences to the users. Classic methods of recommendations, such as collaborative filtering and…

Information Retrieval · Computer Science 2025-01-20 Peiyang Yu , Zeqiu Xu , Jiani Wang , Xiaochuan Xu

Large language models (LLMs) have not only revolutionized the field of natural language processing (NLP) but also have the potential to bring a paradigm shift in many other fields due to their remarkable abilities of language understanding,…

Information Retrieval · Computer Science 2024-10-29 Qi Wang , Jindong Li , Shiqi Wang , Qianli Xing , Runliang Niu , He Kong , Rui Li , Guodong Long , Yi Chang , Chengqi Zhang

Large Language Models (LLMs) have demonstrated remarkable performance across diverse domains, thereby prompting researchers to explore their potential for use in recommendation systems. Initial attempts have leveraged the exceptional…

Information Retrieval · Computer Science 2023-10-18 Keqin Bao , Jizhi Zhang , Yang Zhang , Wenjie Wang , Fuli Feng , Xiangnan He

Recent research has explored using Large Language Models for recommendation tasks by transforming user interaction histories and item metadata into text prompts, then having the LLM produce rankings or recommendations. A promising approach…

Information Retrieval · Computer Science 2025-10-03 Bo Ma , LuYao Liu , Simon Lau , Chandler Yuan , and XueY Cui , Rosie Zhang

Multimedia recommender systems focus on utilizing behavioral information and content information to model user preferences. Typically, it employs pre-trained feature encoders to extract content features, then fuses them with behavioral…

Information Retrieval · Computer Science 2025-01-14 Qile Fan , Penghang Yu , Zhiyi Tan , Bing-Kun Bao , Guanming Lu

The goal of sequential recommendation (SR) is to predict a user's potential interested items based on her/his historical interaction sequences. Most existing sequential recommenders are developed based on ID features, which, despite their…

Information Retrieval · Computer Science 2023-10-24 Jinpeng Wang , Ziyun Zeng , Yunxiao Wang , Yuting Wang , Xingyu Lu , Tianxiang Li , Jun Yuan , Rui Zhang , Hai-Tao Zheng , Shu-Tao Xia

Recent advancements in language models and pre-trained language models like BERT and RoBERTa have revolutionized natural language processing, enabling a deeper understanding of human-like language. In this paper, we explore enhancing…

Information Retrieval · Computer Science 2025-04-15 Ngoc Luyen Le , Marie-Hélène Abel