诊断和克服视频-LLM 中的方向运动致盲
摘要
视频大语言模型 (Video-LLMs) 在时序视频理解方面取得了快速进展，但许多模型在基础感知原语方面仍有缺失：即图像平面上的有符号运动方向。 On simple videos of a single object moving left, right, up, or down, most Video-LLMs perform near chance, with above-chance cases largely attributable to prediction biases rather than genuine direction understanding. We call this failure directional motion blindness. We localize the failure by tracing motion direction information through the Video-LLM pipeline. Motion direction remains linearly accessible from the vision encoder, projector, and LLM hidden states, but the readout fails to bind this signal to the correct verbal answer option, revealing a direction binding gap. Although synthetic motion direction instruction tuning reduces this gap on the source domain, motion direction concept vector analysis shows that visual complexity weakens the signal magnitude and limits out-of-domain generalization. We introduce MoDirect, a dataset family for motion direction instruction tuning and evaluation, and DeltaDirect, a diagnosis-driven, projector-level objective that predicts normalized 2-D motion vectors from adjacent-frame feature deltas. On MoDirect-SynBench, instruction tuning with DeltaDirect improves motion direction accuracy from 25.9% to 85.4%. On MoDirect-RealBench, DeltaDirect improves real-world motion direction accuracy by 21.9 points over the vanilla baseline without real-world tuning data, while preserving standard video-understanding performance. Code: https://github.com/KHU-VLL/DeltaDirect
引用
@article{arxiv.2605.22823,
title = {Which Way Did It Move? Diagnosing and Overcoming Directional Motion Blindness in Video-LLMs},
author = {Jongseo Lee and Hyuntak Lee and Sunghun Kim and Sooa Kim and Jihoon Chung and Jinwoo Choi},
journal= {arXiv preprint arXiv:2605.22823},
year = {2026}
}
备注
Preprint. 59 pages, including appendix. Code: https://github.com/KHU-VLL/DeltaDirect