GeoChain: Multimodal Chain-of-Thought for Geographic Reasoning

Sahiti Yerramilli; Nilay Pande; Rynaa Grover; Jayant Sravan Tamarapalli

GeoChain: Multimodal Chain-of-Thought for Geographic Reasoning

Artificial Intelligence 2025-09-10 v3 Computer Vision and Pattern Recognition Machine Learning

Authors: Sahiti Yerramilli , Nilay Pande , Rynaa Grover , Jayant Sravan Tamarapalli

Abstract

This paper introduces GeoChain, a large-scale benchmark for evaluating step-by-step geographic reasoning in multimodal large language models (MLLMs). Leveraging 1.46 million Mapillary street-level images, GeoChain pairs each image with a 21-step chain-of-thought (CoT) question sequence (over 30 million Q&A pairs). These sequences guide models from coarse attributes to fine-grained localization across four reasoning categories - visual, spatial, cultural, and precise geolocation - annotated by difficulty. Images are also enriched with semantic segmentation (150 classes) and a visual locatability score. Our benchmarking of contemporary MLLMs (GPT-4.1 variants, Claude 3.7, Gemini 2.5 variants) on a diverse 2,088-image subset reveals consistent challenges: models frequently exhibit weaknesses in visual grounding, display erratic reasoning, and struggle to achieve accurate localization, especially as the reasoning complexity escalates. GeoChain offers a robust diagnostic methodology, critical for fostering significant advancements in complex geographic reasoning within MLLMs.

Keywords

geographic reasoning multimodal large language models chain-of-thought visual grounding benchmark street-level images

Cite

@article{arxiv.2506.00785,
  title  = {GeoChain: Multimodal Chain-of-Thought for Geographic Reasoning},
  author = {Sahiti Yerramilli and Nilay Pande and Rynaa Grover and Jayant Sravan Tamarapalli},
  journal= {arXiv preprint arXiv:2506.00785},
  year   = {2025}
}

GeoChain: Multimodal Chain-of-Thought for Geographic Reasoning

Abstract

Keywords

Cite

Related papers