English

Interpretable Perception and Reasoning for Audiovisual Geolocation

Computer Vision and Pattern Recognition 2026-03-09 v1

Abstract

While recent advances in Multimodal Large Language Models (MLLMs) have improved image-based localization, precise global geolocation remains a formidable challenge due to the inherent ambiguity of visual landscapes and the largely untapped potential of auditory cues. In this paper, we introduce Audiovisual Geolocation, a framework designed to resolve geographic ambiguity through interpretable perception and reasoning. We present AVG, a high-quality global-scale video benchmark for geolocation, comprising 20,000 curated clips across 1,000 distinct locations. To address the complexity of audiovisual geolocation, we propose a three-stage framework: (1) a Perception stage that utilizes a mixture-autoregressive sparse autoencoder to decompose noisy audio into semantically grounded "acoustic atoms"; (2) a Multimodal Reasoning stage that employs an MLLM finetuned via Group Relative Policy Optimization (GRPO) to synthesize these atoms with visual features; and (3) a Precision Prediction stage using Riemannian Flow Matching on the S2S^2 manifold. Our experiments demonstrate that our framework significantly outperforms unimodal baselines. These results entail that interpretable perception of the soundscape provides a critical, orthogonal signal that, when coupled with multimodal reasoning, enables high-precision global localization.

Keywords

Cite

@article{arxiv.2603.05708,
  title  = {Interpretable Perception and Reasoning for Audiovisual Geolocation},
  author = {Yiyang Su and Xiaoming Liu},
  journal= {arXiv preprint arXiv:2603.05708},
  year   = {2026}
}
R2 v1 2026-07-01T11:05:48.685Z