English

LaViT: Aligning Latent Visual Thoughts for Multi-modal Reasoning

Computer Vision and Pattern Recognition 2026-01-16 v1 Artificial Intelligence

Abstract

Current multimodal latent reasoning often relies on external supervision (e.g., auxiliary images), ignoring intrinsic visual attention dynamics. In this work, we identify a critical Perception Gap in distillation: student models frequently mimic a teacher's textual output while attending to fundamentally divergent visual regions, effectively relying on language priors rather than grounded perception. To bridge this, we propose LaViT, a framework that aligns latent visual thoughts rather than static embeddings. LaViT compels the student to autoregressively reconstruct the teacher's visual semantics and attention trajectories prior to text generation, employing a curriculum sensory gating mechanism to prevent shortcut learning. Extensive experiments show that LaViT significantly enhances visual grounding, achieving up to +16.9% gains on complex reasoning tasks and enabling a compact 3B model to outperform larger open-source variants and proprietary models like GPT-4o.

Keywords

Cite

@article{arxiv.2601.10129,
  title  = {LaViT: Aligning Latent Visual Thoughts for Multi-modal Reasoning},
  author = {Linquan Wu and Tianxiang Jiang and Yifei Dong and Haoyu Yang and Fengji Zhang and Shichaang Meng and Ai Xuan and Linqi Song and Jacky Keung},
  journal= {arXiv preprint arXiv:2601.10129},
  year   = {2026}
}
R2 v1 2026-07-01T09:05:24.515Z