TangramSR: Can Vision-Language Models Reason in Continuous Geometric Space?

2026-02-05

AI Summary

Existing vision-language models (VLMs) exhibit limited reasoning capabilities in continuous geometric spaces, struggling with tasks requiring iterative spatial optimization such as tangram puzzles. Inspired by human trial-and-error and corrective cognitive mechanisms, this work proposes a training-free, test-time self-optimization framework that recursively refines geometric predictions through in-context learning, a geometric consistency verifier, and a reward-guided feedback loop. This approach formalizes human-like spatial reasoning into a computable, training-agnostic agent for the first time. Evaluated on the medium triangle tangram task, the method improves Intersection over Union (IoU) from 0.63 to 0.932, substantially narrowing the performance gap between VLMs and human-level continuous geometric reasoning.

Technology Category

• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Computer Vision: Large Vision Models
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Humans excel at spatial reasoning tasks like Tangram puzzle assembly through cognitive processes involving mental rotation, iterative refinement, and visual feedback. Inspired by how humans solve Tangram puzzles through trial-and-error, observation, and correction, we design a framework that models these human cognitive mechanisms. However, comprehensive experiments across five representative Vision-Language Models (VLMs) reveal systematic failures in continuous geometric reasoning: average IoU of only 0.41 on single-piece tasks, dropping to 0.23 on two-piece composition, far below human performance where children can complete Tangram tasks successfully. This paper addresses a fundamental challenge in self-improving AI: can models iteratively refine their predictions at test time without parameter updates? We introduce a test-time self-refinement framework that combines in-context learning (ICL) with reward-guided feedback loops, inspired by human cognitive processes. Our training-free verifier-refiner agent applies recursive refinement loops that iteratively self-refine predictions based on geometric consistency feedback, achieving IoU improvements from 0.63 to 0.932 on medium-triangle cases without any model retraining. This demonstrates that incorporating human-inspired iterative refinement mechanisms through ICL and reward loops can substantially enhance geometric reasoning in VLMs, moving self-improving AI from promise to practice in continuous spatial domains. Our work is available at this anonymous link https://anonymous.4open.science/r/TangramVLM-F582/.

Problem

• spatial reasoning
• continuous geometric space
• Vision-Language Models
• Tangram puzzle
• geometric reasoning

Innovation

• test-time self-refinement
• in-context learning
• geometric reasoning
• vision-language models
• reward-guided feedback