English
Related papers

Related papers: MedXplain-VQA: Multi-Component Explainable Medical…

200 papers

Our society increasingly depends on intelligent systems to solve complex problems, ranging from recommender systems suggesting the next movie to watch to AI models assisting in medical diagnoses for hospitalized patients. With the iterative…

Human-Computer Interaction · Computer Science 2025-07-15 Angelos Chatzimparmpas

Although explainability is essential in the clinical diagnosis, most deep learning models still function as black boxes without elucidating their decision-making process. In this study, we investigate the explainable model development that…

Computer Vision and Pattern Recognition · Computer Science 2024-09-20 Yunhe Gao , Difei Gu , Mu Zhou , Dimitris Metaxas

In Medical question-answering (QA) tasks, the need for effective systems is pivotal in delivering accurate responses to intricate medical queries. However, existing approaches often struggle to grasp the intricate logical structures and…

Computation and Language · Computer Science 2024-10-22 Aizan Zafar , Kshitij Mishra , Asif Ekbal

Visual question answering (VQA) for crop disease analysis requires accurate visual understanding and reliable language generation. In this work, we present a lightweight and explainable vision-language framework for crop and disease…

Computer Vision and Pattern Recognition · Computer Science 2026-03-10 Md. Zahid Hossain , Most. Sharmin Sultana Samu , Md. Rakibul Islam , Md. Siam Ansary

Vision-Language Models (VLMs) have enabled interpretable medical diagnosis by integrating visual perception with linguistic reasoning. Yet, existing medical chain-of-thought (CoT) models lack explicit mechanisms to represent and enforce…

Artificial Intelligence · Computer Science 2026-05-29 Jianxin Lin , Chunzheng Zhu , Peter J. Kneuertz , Yunfei Bai , Yuan Xue

Most recent state-of-the-art Visual Question Answering (VQA) systems are opaque black boxes that are only trained to fit the answer distribution given the question and visual content. As a result, these systems frequently take shortcuts,…

Computer Vision and Pattern Recognition · Computer Science 2020-06-30 Jialin Wu , Liyan Chen , Raymond J. Mooney

The Visual Question Answering (VQA) task combines challenges for processing data with both Visual and Linguistic processing, to answer basic `common sense' questions about given images. Given an image and a question in natural language, the…

Computer Vision and Pattern Recognition · Computer Science 2020-12-24 Yash Srivastava , Vaishnav Murali , Shiv Ram Dubey , Snehasis Mukherjee

We propose the task of free-form and open-ended Visual Question Answering (VQA). Given an image and a natural language question about the image, the task is to provide an accurate natural language answer. Mirroring real-world scenarios,…

Computation and Language · Computer Science 2016-10-28 Aishwarya Agrawal , Jiasen Lu , Stanislaw Antol , Margaret Mitchell , C. Lawrence Zitnick , Dhruv Batra , Devi Parikh

The recent development of data-driven AI promises to automate medical diagnosis; however, most AI functions as 'black boxes' to physicians with limited computational knowledge. Using medical imaging as a point of departure, we conducted…

Human-Computer Interaction · Computer Science 2020-01-22 Yao Xie , Melody Chen , David Kao , Ge Gao , Xiang 'Anthony' Chen

Despite the promise of foundation models in medical AI, current systems remain limited - they are modality-specific and lack transparent reasoning processes, hindering clinical adoption. To address this gap, we present EVLF-FM, a multimodal…

The Medico 2025 challenge addresses Visual Question Answering (VQA) for Gastrointestinal (GI) imaging, organized as part of the MediaEval task series. The challenge focuses on developing Explainable Artificial Intelligence (XAI) models that…

Computer Vision and Pattern Recognition · Computer Science 2025-08-15 Sushant Gautam , Vajira Thambawita , Michael Riegler , Pål Halvorsen , Steven Hicks

In healthcare and medical diagnostics, Visual Question Answering (VQA) mayemergeasapivotal tool in scenarios where analysis of intricate medical images becomes critical for accurate diagnoses. Current text-based VQA systems limit their…

Computer Vision and Pattern Recognition · Computer Science 2024-07-17 Tonmoy Rajkhowa , Amartya Roy Chowdhury , Sankalp Nagaonkar , Achyut Mani Tripathi

Medical vision language pre-training (VLP) has emerged as a frontier of research, enabling zero-shot pathological recognition by comparing the query image with the textual descriptions for each disease. Due to the complex semantics of…

Computer Vision and Pattern Recognition · Computer Science 2024-04-02 Vu Minh Hieu Phan , Yutong Xie , Yuankai Qi , Lingqiao Liu , Liyang Liu , Bowen Zhang , Zhibin Liao , Qi Wu , Minh-Son To , Johan W. Verjans

AI tools in pathology have improved screening throughput, standardized quantification, and revealed prognostic patterns that inform treatment. However, adoption remains limited because most systems still lack the human-readable reasoning…

Artificial Intelligence · Computer Science 2025-11-18 Yunqi Hong , Johnson Kao , Liam Edwards , Nein-Tzu Liu , Chung-Yen Huang , Alex Oliveira-Kowaleski , Cho-Jui Hsieh , Neil Y. C. Lin

Vision-language models (VLMs) are increasingly adapted through domain-specific fine-tuning, yet it remains unclear whether this improves reasoning beyond superficial visual cues, particularly in high-stakes domains like medicine. We…

Computer Vision and Pattern Recognition · Computer Science 2026-04-14 Oliver McLaughlin , Daniel Shubin , Carsten Eickhoff , Ritambhara Singh , William Rudman , Michal Golovanevsky

Explainable AI aims to render model behavior understandable by humans, which can be seen as an intermediate step in extracting causal relations from correlative patterns. Due to the high risk of possible fatal decisions in image-based…

Computer Vision and Pattern Recognition · Computer Science 2023-06-16 Lukas Klein , João B. S. Carvalho , Mennatallah El-Assady , Paolo Penna , Joachim M. Buhmann , Paul F. Jaeger

We present NeuroQA, a large-scale benchmark for visual question answering in 3D brain magnetic resonance imaging (MRI), with 56,953 QA pairs from 12,977 subjects across 12 datasets. It spans ages 5-104 and five clinical domains:…

Vision-Language Models (VLMs) have demonstrated remarkable progress in multimodal understanding, yet their capabilities for scientific reasoning remain inadequately assessed. Current multimodal benchmarks predominantly evaluate generic…

Computer Vision and Pattern Recognition · Computer Science 2025-06-18 Ai Jian , Weijie Qiu , Xiaokun Wang , Peiyu Wang , Yunzhuo Hao , Jiangbo Pei , Yichen Wei , Yi Peng , Xuchen Song

As Vision-Language Models (VLMs) increasingly gain traction in medical applications, clinicians are progressively expecting AI systems not only to generate textual diagnoses but also to produce corresponding medical images that integrate…

Computer Vision and Pattern Recognition · Computer Science 2025-11-19 Junjie Yang , Yuhao Yan , Gang Wu , Yuxuan Wang , Ruoyu Liang , Xinjie Jiang , Xiang Wan , Fenglei Fan , Yongquan Zhang , Feiwei Qin , Changmiao Wang

Accurate diagnosis of skin diseases remains a significant challenge due to the complex and diverse visual features present in dermatoscopic images, often compounded by a lack of interpretability in existing purely visual diagnostic models.…

Computer Vision and Pattern Recognition · Computer Science 2025-08-12 Kexin Yu , Zihan Xu , Jialei Xie , Carter Adams