English
Related papers

Related papers: Lost in Modality: Evaluating the Effectiveness of …

200 papers

Connecting text and visual modalities plays an essential role in generative intelligence. For this reason, inspired by the success of large language models, significant research efforts are being devoted to the development of Multimodal…

Computer Vision and Pattern Recognition · Computer Science 2024-06-07 Davide Caffagni , Federico Cocchi , Luca Barsellotti , Nicholas Moratelli , Sara Sarto , Lorenzo Baraldi , Lorenzo Baraldi , Marcella Cornia , Rita Cucchiara

Membership Inference Attacks (MIAs) are widely used to quantify training data memorization and assess privacy risks. Standard evaluation requires repeated retraining, which is computationally costly for large models. One-run methods (single…

Machine Learning · Computer Science 2026-02-06 Mathieu Even , Clément Berenfeld , Linus Bleistein , Tudor Cebere , Julie Josse , Aurélien Bellet

Social media influence campaigns pose significant challenges to public discourse and democracy. Traditional detection methods fall short due to the complexity and dynamic nature of social media. Addressing this, we propose a novel detection…

Social and Information Networks · Computer Science 2023-11-15 Luca Luceri , Eric Boniardi , Emilio Ferrara

Federated Large Language Models (FedLLMs) enable multiple parties to collaboratively fine-tune LLMs without sharing raw data, addressing challenges of limited resources and privacy concerns. Despite data localization, shared gradients can…

Machine Learning · Computer Science 2026-04-24 Guilin Deng , Silong Chen , Yuchuan Luo , Yi Liu , Songlei Wang , Zhiping Cai , Lin Liu , Xiaohua Jia , Shaojing Fu

As audio-visual multi-modal large language models (MLLMs) are increasingly deployed in safety-critical applications, understanding their vulnerabilities is crucial. To this end, we introduce Multi-Modal Typography, a systematic study…

Computer Vision and Pattern Recognition · Computer Science 2026-04-07 Tianle Chen , Deepti Ghadiyaram

This survey and application guide to multimodal large language models(MLLMs) explores the rapidly developing field of MLLMs, examining their architectures, applications, and impact on AI and Generative Models. Starting with foundational…

Artificial Intelligence · Computer Science 2025-12-02 Chia Xin Liang , Pu Tian , Caitlyn Heqi Yin , Yao Yua , Wei An-Hou , Li Ming , Xinyuan Song , Tianyang Wang , Ziqian Bi , Ming Liu

Large Language Models (LLMs), benefiting from the auto-regressive modelling approach performed on massive unannotated texts corpora, demonstrates powerful perceptual and reasoning capabilities. However, as for extending auto-regressive…

Computer Vision and Pattern Recognition · Computer Science 2024-09-24 Tianshuo Peng , Zuchao Li , Lefei Zhang , Hai Zhao , Ping Wang , Bo Du

A fundamental issue in deep learning has been adversarial robustness. As these systems have scaled, such issues have persisted. Currently, large language models (LLMs) with billions of parameters suffer from adversarial attacks just like…

Machine Learning · Computer Science 2025-02-11 Brian Formento , Chuan Sheng Foo , See-Kiong Ng

Membership inference attacks (MIAs) have become the standard tool for evaluating privacy leakage in machine learning (ML). Among them, the Likelihood-Ratio Attack (LiRA) is widely regarded as the state of the art when sufficient shadow…

Cryptography and Security · Computer Science 2026-03-10 Najeeb Jebreel , Mona Khalil , David Sánchez , Josep Domingo-Ferrer

The advent of Large Language Models (LLMs) has garnered significant popularity and wielded immense power across various domains within Natural Language Processing (NLP). While their capabilities are undeniably impressive, it is crucial to…

Machine Learning · Computer Science 2024-07-31 Sara Abdali , Jia He , CJ Barberan , Richard Anarfi

As large language models (LLMs) are trained on increasingly opaque corpora, membership inference attacks (MIAs) have been proposed to audit whether copyrighted texts were used during training, despite growing concerns about their…

Cryptography and Security · Computer Science 2026-01-21 Murat Bilgehan Ertan , Emirhan Böge , Min Chen , Kaleel Mahmood , Marten van Dijk

Recently, large language models (LLMs) have been gaining a lot of interest due to their adaptability and extensibility in emerging applications, including communication networks. It is anticipated that ZSM networks will be able to support…

Cryptography and Security · Computer Science 2025-01-07 Sunder Ali Khowaja , Parus Khuwaja , Kapal Dev , Hussam Al Hamadi , Engin Zeydan

Multimodal Large Language Models (MLLMs) have demonstrated strong cross-modal reasoning capabilities, yet their potential for vision-only tasks remains underexplored. We investigate MLLMs as training-free similarity estimators for…

Computer Vision and Pattern Recognition · Computer Science 2026-04-16 Bahey Tharwat , Giorgos Kordopatis-Zilos , Pavel Suma , Ian Reid , Giorgos Tolias

Large Reasoning Models (LRMs) have rapidly gained prominence for their strong performance in solving complex tasks. Many modern black-box LRMs expose the intermediate reasoning traces through APIs to improve transparency (e.g., Gemini-2.5…

Cryptography and Security · Computer Science 2026-01-21 Ruihan Hu , Yu-Ming Shang , Wei Luo , Ye Tao , Xi Zhang

Vision language models (VLMs) extend the reasoning capabilities of large language models (LLMs) to cross-modal settings, yet remain highly vulnerable to multimodal jailbreak attacks. Existing defenses predominantly rely on safety…

Artificial Intelligence · Computer Science 2026-04-14 Mengxuan Wang , Yuxin Chen , Gang Xu , Tao He , Hongjie Jiang , Ming Li

Membership inference attacks (MIAs) are used to test practical privacy of machine learning models. MIAs complement formal guarantees from differential privacy (DP) under a more realistic adversary model. We analyse MIA vulnerability of…

Cryptography and Security · Computer Science 2026-02-03 Marlon Tobaben , Hibiki Ito , Joonas Jälkö , Yuan He , Antti Honkela

Vision-language models (VLMs) allow to embed texts and images in a shared representation space. However, it has been shown that these models are subject to a modality gap phenomenon meaning there exists a clear separation between the…

Computer Vision and Pattern Recognition · Computer Science 2025-05-07 François Role , Sébastien Meyer , Victor Amblard

Large language models (LLMs) can leak sensitive training data through memorization and membership inference attacks. Prior work has primarily focused on strong adversarial assumptions, including attacker access to entire samples or long,…

Machine Learning · Computer Science 2025-05-21 Lucas Rosenblatt , Bin Han , Robert Wolfe , Bill Howe

Multimodal Large Language Models (MLLMs) utilize multimodal contexts consisting of text, images, or videos to solve various multimodal tasks. However, we find that changing the order of multimodal input can cause the model's performance to…

Artificial Intelligence · Computer Science 2024-10-23 Zhijie Tan , Xu Chu , Weiping Li , Tong Mo

Membership Inference Attacks (MIAs) are currently a dominant approach for evaluating privacy in machine learning applications. Despite their significance in identifying records belonging to the training dataset, several concerns remain…

Machine Learning · Computer Science 2026-01-23 Cristina Pêra , Tânia Carvalho , Maxime Cordy , Luís Antunes