Search papers, labs, and topics across Lattice.
This paper introduces the JigShape benchmark to evaluate visual-geometric reasoning in vision-language models (VLMs) using jigsaw puzzles with interlocking pieces that enforce strong local compatibility. The study reveals that while zero-shot VLMs struggle with geometric reasoning鈥攐nly GPT-5.5 surpasses random performance on simple 4x4 puzzles鈥攕upervised fine-tuning fails to scale, with models collapsing to chance levels on larger grids. These findings highlight a significant "scaling cliff" in current architectures, indicating a critical gap in their ability to handle complex geometric constraints as puzzle size increases.
Zero-shot VLMs falter at geometric reasoning, with only one out of five models surpassing random performance on basic jigsaw puzzles, revealing a scaling cliff in their capabilities.
Jigsaw puzzle solving requires jointly reasoning about visual content and geometric constraints, yet existing benchmarks use rectangular cuts that create ambiguous ground truth in texture-repeated regions. We introduce \textit{\ours{}}, a benchmark with tab-and-blank interlocking pieces where geometric constraints provide strong local compatibility requirements that, combined with visual content, yield unambiguous ground truth. Across 95K instances at four grid densities (4$\times$4 to 16$\times$16), we find that \textbf{zero-shot VLMs largely lack geometric reasoning}: only one of five frontier models (GPT-5.5) exceeds random baseline on 4$\times$4 puzzles, while all others perform at chance level. While supervised fine-tuning achieves $>$97\% on 4$\times$4, \textbf{all models collapse on larger grids}: GPT-5.5 drops from 70\% to near-random on 8$\times$8, and even fine-tuned models fall below 5\% on 12$\times$12. This ``scaling cliff''suggests current architectures cannot maintain consistent constraint satisfaction as the number of pieces increases. \ours{} establishes scalable geometric reasoning as an open challenge for vision-language models.
