English
Related papers

Related papers: Evaluating Cell Type Inference in Vision Language …

200 papers

Visual-language pre-training has achieved remarkable success in many multi-modal tasks, largely attributed to the availability of large-scale image-text datasets. In this work, we demonstrate that Multi-modal Large Language Models (MLLMs)…

Computer Vision and Pattern Recognition · Computer Science 2024-03-14 Yanqing Liu , Kai Wang , Wenqi Shao , Ping Luo , Yu Qiao , Mike Zheng Shou , Kaipeng Zhang , Yang You

Human age estimation from facial images represents a challenging computer vision task with significant applications in biometrics, healthcare, and human-computer interaction. While traditional deep learning approaches require extensive…

Computer Vision and Pattern Recognition · Computer Science 2026-03-30 Rakib Hossain Sajib , Md Kishor Morol , Rajan Das Gupta , Mohammad Sakib Mahmood , Shuvra Smaran Das

Image-based visual-language (I-VL) pre-training has shown great success for learning joint visual-textual representations from large-scale web data, revealing remarkable ability for zero-shot generalisation. This paper presents a simple but…

Computer Vision and Pattern Recognition · Computer Science 2022-07-18 Chen Ju , Tengda Han , Kunhao Zheng , Ya Zhang , Weidi Xie

Integrating image and text data through multi-modal learning has emerged as a new approach in medical imaging research, following its successful deployment in computer vision. While considerable efforts have been dedicated to establishing…

Computer Vision and Pattern Recognition · Computer Science 2024-09-09 Fereshteh Shakeri , Yunshi Huang , Julio Silva-Rodríguez , Houda Bahig , An Tang , Jose Dolz , Ismail Ben Ayed

Recent progress in large language models (LLMs) has shown strong potential for multimodal reasoning beyond natural language. In this work, we explore the use of a fine-tuned Vision-Language Model (VLM), based on LLaMA 3.2, for classifying…

Machine Learning · Computer Science 2025-08-28 Dikshant Sagar , Kaiwen Yu , Alejandro Yankelevich , Jianming Bian , Pierre Baldi

Automation in agriculture plays a vital role in addressing challenges related to crop monitoring and disease management, particularly through early detection systems. This study investigates the effectiveness of combining multimodal Large…

Computer Vision and Pattern Recognition · Computer Science 2025-04-30 Konstantinos I. Roumeliotis , Ranjan Sapkota , Manoj Karkee , Nikolaos D. Tselikas , Dimitrios K. Nasiopoulos

Vision and Language (VL) models have demonstrated remarkable zero-shot performance in a variety of tasks. However, some aspects of complex language understanding still remain a challenge. We introduce the collective notion of Structured…

Computer Vision and Pattern Recognition · Computer Science 2023-06-01 Sivan Doveh , Assaf Arbelle , Sivan Harary , Rameswar Panda , Roei Herzig , Eli Schwartz , Donghyun Kim , Raja Giryes , Rogerio Feris , Shimon Ullman , Leonid Karlinsky

A recent advancement in Multimodal Large Language Models (MLLMs) research is the emergence of "reasoning MLLMs" that offer explicit control over their internal thinking processes (normally referred as the "thinking mode") alongside the…

Computation and Language · Computer Science 2025-11-06 Jindong Hong , Tianjie Chen , Lingjie Luo , Chuanyang Zheng , Ting Xu , Haibao Yu , Jianing Qiu , Qianzhong Chen , Suning Huang , Yan Xu , Yong Gui , Yijun He , Jiankai Sun

Patent figure classification facilitates faceted search in patent retrieval systems, enabling efficient prior art search. Existing approaches have explored patent figure classification for only a single aspect and for aspects with a limited…

Information Retrieval · Computer Science 2025-01-23 Sushil Awale , Eric Müller-Budack , Ralph Ewerth

Vision-Language Models (VLMs) have demonstrated impressive capabilities across a range of tasks, yet concerns about their potential biases exist. This work investigates the extent to which prominent VLMs exhibit cultural biases by…

Computer Vision and Pattern Recognition · Computer Science 2025-08-01 Ram Mohan Rao Kadiyala , Siddhant Gupta , Jebish Purbey , Srishti Yadav , Suman Debnath , Alejandro Salamanca , Desmond Elliott

Previous learning-based vulnerability detection methods relied on either medium-sized pre-trained models or smaller neural networks from scratch. Recent advancements in Large Pre-Trained Language Models (LLMs) have showcased remarkable…

Software Engineering · Computer Science 2024-01-30 Xin Zhou , Ting Zhang , David Lo

Large Language Model (LLM)-based Vision-Language Models (VLMs) have substantially extended the boundaries of visual understanding capabilities. However, their high computational demands hinder deployment on resource-constrained edge…

Computer Vision and Pattern Recognition · Computer Science 2025-06-10 Haotong Qin , Cheng Hu , Michele Magno

Vision-Language Models (VLMs) have demonstrated impressive capabilities in zero-shot action recognition by learning to associate video embeddings with class embeddings. However, a significant challenge arises when relying solely on action…

Computer Vision and Pattern Recognition · Computer Science 2025-11-04 Yehna Kim , Young-Eun Kim , Seong-Whan Lee

When answering questions about images, humans naturally point, label, and draw to explain their reasoning. In contrast, modern vision-language models (VLMs) such as Gemini-3-Pro and GPT-5 only respond with text, which can be difficult for…

Computer Vision and Pattern Recognition · Computer Science 2026-04-29 Brandon Collins , Logan Bolton , Hung Huy Nguyen , Mohammad Reza Taesiri , Trung Bui , Anh Totti Nguyen

Pre-trained vision-language models (VLMs) have shown remarkable generalization capabilities via prompting, which leverages VLMs as knowledge bases to extract information beneficial for downstream tasks. However, existing methods primarily…

Computer Vision and Pattern Recognition · Computer Science 2024-04-25 Xiaoyu Qiu , Hao Feng , Yuechen Wang , Wengang Zhou , Houqiang Li

Large language models (LLMs) have been effectively used for many computer vision tasks, including image classification. In this paper, we present a simple yet effective approach for zero-shot image classification using multimodal LLMs.…

Computer Vision and Pattern Recognition · Computer Science 2025-06-27 Abdelrahman Abdelhamed , Mahmoud Afifi , Alec Go

Vision-language models (VLMs) can learn high-quality representations from a large-scale training dataset of image-text pairs. Prompt learning is a popular approach to fine-tuning VLM to adapt them to downstream tasks. Despite the satisfying…

Computer Vision and Pattern Recognition · Computer Science 2024-12-31 Zhifang Zhang , Yuwei Niu , Xin Liu , Beibei Li

In medical healthcare, obtaining detailed annotations is challenging, highlighting the need for robust Vision-Language Models (VLMs). Pretrained VLMs enable fine-tuning on small datasets or zero-shot inference, achieving performance…

Computer Vision and Pattern Recognition · Computer Science 2025-12-03 Phuc Pham , Nhu Pham , Ngoc Quoc Ly

Classifying scanned documents is a challenging problem that involves image, layout, and text analysis for document understanding. Nevertheless, for certain benchmark datasets, notably RVL-CDIP, the state of the art is closing in to…

Computer Vision and Pattern Recognition · Computer Science 2024-12-19 Anna Scius-Bertrand , Michael Jungo , Lars Vögtlin , Jean-Marc Spat , Andreas Fischer

Language provides a natural interface to specify and evaluate performance on visual tasks. To realize this possibility, vision language models (VLMs) must successfully integrate visual and linguistic information. Our work compares VLMs to a…

Computer Vision and Pattern Recognition · Computer Science 2025-06-10 Stephanie Fu , Tyler Bonnen , Devin Guillory , Trevor Darrell