English
Related papers

Related papers: Lost in Modality: Evaluating the Effectiveness of …

200 papers

Membership Inference Attacks (MIAs) have recently been employed to determine whether a specific text was part of the pre-training data of Large Language Models (LLMs). However, existing methods often misinfer non-members as members, leading…

Machine Learning · Computer Science 2025-05-07 Saleh Zare Zade , Yao Qiang , Xiangyu Zhou , Hui Zhu , Mohammad Amin Roshani , Prashant Khanduri , Dongxiao Zhu

Instruction-tuned large language models (LLMs) have demonstrated promising zero-shot generalization capabilities across various downstream tasks. Recent research has introduced multimodal capabilities to LLMs by integrating independently…

Computation and Language · Computer Science 2023-11-29 Utsav Garg , Erhan Bas

Recent Multimodal Large Language Models (MLLMs) have demonstrated significant progress in perceiving and reasoning over multimodal inquiries, ushering in a new research era for foundation models. However, vision-language misalignment in…

Computer Vision and Pattern Recognition · Computer Science 2026-05-18 Wei-Yao Wang , Zhao Wang , Helen Suzuki , Yoshiyuki Kobayashi

Multi-Modal Language Models (MLLMs) have transformed artificial intelligence by combining visual and text data, making applications like image captioning, visual question answering, and multi-modal content creation possible. This ability to…

Cryptography and Security · Computer Science 2024-11-11 Pete Janowczyk , Linda Laurier , Ave Giulietta , Arlo Octavia , Meade Cleti

The proliferation of powerful Text-to-Video (T2V) models, trained on massive web-scale datasets, raises urgent concerns about copyright and privacy violations. Membership inference attacks (MIAs) provide a principled tool for auditing such…

Cryptography and Security · Computer Science 2026-01-19 Li Wang , Wenyu Chen , Ning Yu , Zheng Li , Shanqing Guo

Vision-Language Models (VLMs) extend large language models with visual reasoning, but their multimodal design also introduces new, underexplored vulnerabilities. Existing multimodal red-teaming methods largely rely on brittle templates,…

Cryptography and Security · Computer Science 2026-05-27 Qilin Liao , Anamika Lochab , Ruqi Zhang

The rise of generative image models leads to privacy concerns when it comes to the huge datasets used to train such models. This paper investigates the possibility of inferring if a set of face images was used for fine-tuning a Latent…

Computer Vision and Pattern Recognition · Computer Science 2025-03-05 Lauritz Christian Holme , Anton Mosquera Storgaard , Siavash Arjomand Bigdeli

The increase in parameter size of multimodal large language models (MLLMs) introduces significant capabilities, particularly in-context learning, where MLLMs enhance task performance without updating pre-trained parameters. This…

Computation and Language · Computer Science 2024-11-13 Yang Luo , Zangwei Zheng , Zirui Zhu , Yang You

Membership inference attacks (MIAs) aim to determine whether specific data were used to train a model. While extensively studied on classification models, their impact on time series forecasting remains largely unexplored. We address this…

Machine Learning · Computer Science 2026-02-13 Nicolas Johansson , Tobias Olsson , Daniel Nilsson , Johan Östman , Fazeleh Hoseini

While Vision-Language Models (VLMs) and Multimodal Large Language Models (MLLMs) have shown strong generalisation in detecting image and video deepfakes, their use for audio deepfake detection remains largely unexplored. In this work, we…

Sound · Computer Science 2026-01-05 Akanksha Chuchra , Shukesh Reddy , Sudeepta Mishra , Abhijit Das , Abhinav Dhall

Multimodal large language models (MLLMs) exhibit a pronounced preference for textual inputs when processing vision-language data, limiting their ability to reason effectively from visual evidence. Unlike prior studies that attribute this…

Artificial Intelligence · Computer Science 2026-04-21 Xinhan Zheng , Huyu Wu , Xueting Wang , Duo Su , Haiyun Jiang

Vision-language models (VLMs) are trained on large-scale image-text corpora that may contain private, copyrighted, or otherwise sensitive data, motivating membership inference as a tool for training-data auditing. This is especially…

Computer Vision and Pattern Recognition · Computer Science 2026-05-14 Hongyi Tang , Zhihao Zhu , Yi Yang

As large language models (LLMs) become progressively more embedded in clinical decision-support, documentation, and patient-information systems, ensuring their privacy and trustworthiness has emerged as an imperative challenge for the…

Cryptography and Security · Computer Science 2025-10-22 Alexander Nemecek , Zebin Yun , Zahra Rahmani , Yaniv Harel , Vipin Chaudhary , Mahmood Sharif , Erman Ayday

Multimodal Large Language Models (MLLMs) have demonstrated exceptional performance in artificial intelligence by facilitating integrated understanding across diverse modalities, including text, images, video, audio, and speech. However,…

Computer Vision and Pattern Recognition · Computer Science 2025-03-19 Chengze Jiang , Zhuangzhuang Wang , Minjing Dong , Jie Gui

Multimodal large language models (MLLMs) have emerged as pivotal tools in enhancing human-computer interaction. In this paper we focus on the application of MLLMs in the field of graphical user interface (GUI) elements structuring, where…

Computer Vision and Pattern Recognition · Computer Science 2025-09-03 Yi Xu , Yesheng Zhang , Jiajia Liu , Jingdong Chen

The integration of Large Language Models (LLMs) and Multi-modal Large Language Models (MLLMs) into mobile GUI agents has significantly enhanced user efficiency and experience. However, this advancement also introduces potential security…

Cryptography and Security · Computer Science 2025-03-18 Yulong Yang , Xinshan Yang , Shuaidong Li , Chenhao Lin , Zhengyu Zhao , Chao Shen , Tianwei Zhang

Membership inference attacks (MIAs) against machine learning (ML) models aim to determine whether a given data point was part of the model training data. These attacks may pose significant privacy risks to individuals whose sensitive data…

Cryptography and Security · Computer Science 2025-11-24 Mona Khalil , Alberto Blanco-Justicia , Najeeb Jebreel , Josep Domingo-Ferrer

Recent studies reveal that integrating new modalities into Large Language Models (LLMs), such as Vision-Language Models (VLMs), creates a new attack surface that bypasses existing safety training techniques like Supervised Fine-tuning (SFT)…

Computation and Language · Computer Science 2025-10-15 Trishna Chakraborty , Erfan Shayegani , Zikui Cai , Nael Abu-Ghazaleh , M. Salman Asif , Yue Dong , Amit K. Roy-Chowdhury , Chengyu Song

Membership inference attacks (MIA) can reveal whether a particular data point was part of the training dataset, potentially exposing sensitive information about individuals. This article provides theoretical guarantees by exploring the…

Machine Learning · Statistics 2025-10-08 Eric Aubinais , Elisabeth Gassiat , Pablo Piantanida

Large Language Models (LLMs) have recently emerged as a focal point of research and application, driven by their unprecedented ability to understand and generate text with human-like quality. Even more recently, LLMs have been extended into…

Computation and Language · Computer Science 2024-04-03 Kilian Carolan , Laura Fennelly , Alan F. Smeaton
‹ Prev 1 3 4 5 6 7 10 Next ›