English

JigShape: Evaluating Visual-Geometric Reasoning in VLMs through Jigsaw Puzzles

Computer Vision and Pattern Recognition 2026-07-30 v1 Artificial Intelligence

Abstract

Jigsaw puzzle solving requires jointly reasoning about visual content and geometric constraints, yet existing benchmarks use rectangular cuts that create ambiguous ground truth in texture-repeated regions. We introduce \textit{\ours{}}, a benchmark with tab-and-blank interlocking pieces where geometric constraints provide strong local compatibility requirements that, combined with visual content, yield unambiguous ground truth. Across 95K instances at four grid densities (4×\times4 to 16×\times16), we find that \textbf{zero-shot VLMs largely lack geometric reasoning}: only one of five frontier models (GPT-5.5) exceeds random baseline on 4×\times4 puzzles, while all others perform at chance level. While supervised fine-tuning achieves >>97\% on 4×\times4, \textbf{all models collapse on larger grids}: GPT-5.5 drops from 70\% to near-random on 8×\times8, and even fine-tuned models fall below 5\% on 12×\times12. This ``scaling cliff'' suggests current architectures cannot maintain consistent constraint satisfaction as the number of pieces increases. \ours{} establishes scalable geometric reasoning as an open challenge for vision-language models.

Cite

@article{arxiv.2607.27670,
  title  = {JigShape: Evaluating Visual-Geometric Reasoning in VLMs through Jigsaw Puzzles},
  author = {Shawn Li and Wei Yang and Jike Zhong and Jiate Li and Jiawei Yang and You Qin and Ryan Rossi and Franck Dernoncourt and Roger Zimmermann and Yue Wang and Zhengzhong Tu and Vicente Ordonez and Mohit Bansal and Yue Zhao},
  journal= {arXiv preprint arXiv:2607.27670},
  year   = {2026}
}