English

Leveraging large multimodal models for audio-video deepfake detection: a pilot study

Sound 2026-03-02 v1 Computer Vision and Pattern Recognition

Abstract

Audio-visual deepfake detection (AVD) is increasingly important as modern generators can fabricate convincing speech and video. Most current multimodal detectors are small, task-specific models: they work well on curated tests but scale poorly and generalize weakly across domains. We introduce AV-LMMDetect, a supervised fine-tuned (SFT) large multimodal model that casts AVD as a prompted yes/no classification - "Is this video real or fake?". Built on Qwen 2.5 Omni, it jointly analyzes audio and visual streams for deepfake detection and is trained in two stages: lightweight LoRA alignment followed by audio-visual encoder full fine-tuning. On FakeAVCeleb and Mavos-DD, AV-LMMDetect matches or surpasses prior methods and sets a new state of the art on Mavos-DD datasets.

Keywords

Cite

@article{arxiv.2602.23393,
  title  = {Leveraging large multimodal models for audio-video deepfake detection: a pilot study},
  author = {Songjun Cao and Yuqi Li and Yunpeng Luo and Jianjun Yin and Long Ma},
  journal= {arXiv preprint arXiv:2602.23393},
  year   = {2026}
}

Comments

5pages,ICASSP2026

R2 v1 2026-07-01T10:54:28.879Z