Can Machines Really See Objects in Images? A Study Based on Syntactic Distance and Visual Self-Referential Instances

2026-06-28

AI Summary

This study investigates whether visual models genuinely comprehend image semantics or merely rely on local statistical cues. To this end, the authors introduce a “syntactic distance” metric to quantify global semantic differences between classes and design a visual self-reference task: closed squares and their single-pixel perturbed variants are embedded within maximum-variance binary noise, effectively nullifying local cues. Experiments with ResNet and Vision Transformer architectures reveal that model accuracy abruptly drops to random chance when image scale exceeds a critical threshold; larger models or datasets only delay this collapse, and models with global attention mechanisms fail even earlier. These findings demonstrate that current vision models fundamentally lack the capacity to understand global structural relationships in images.

Abstract

Can a vision model truly see an object, or does it only fit surface-level visual cues? Following Wittgenstein's view that the limits of language are the limits of the world, we view a model's recognition ability as bounded by the descriptive system it has learned. In current vision models, this system is often realized through learned feature representations that exploit local statistical cues. We therefore ask whether a model can still classify correctly when such local cues provide no stable basis for distinction. We formalize this question with syntactic distance, which measures class separability through the symmetry of the operations mapping one class to the other: positive distance exposes exploitable local features, whereas zero distance requires global semantics rather than local rules. We construct a visual self-referential task in maximum-variance binary noise: positive samples contain a closed square, while negative samples contain an otherwise identical square with one flipped boundary pixel. The two classes differ in global semantics but have zero syntactic distance, making local statistical shortcuts unreliable. Experiments on ResNets and Vision Transformers reveal a consistent phase-transition phenomenon, with accuracy collapsing to random guessing once the image scale crosses a critical point and does not recover within the tested range. Larger training sets and models only delay this collapse, while globally attentive ViTs reach it earlier. These results reveal a structural capability boundary of current architectures on global-concept tasks, suggesting that general intelligence may require creating new language, not reusing an existing one.

Problem

• syntactic distance
• visual self-referential instances
• global semantics
• vision models
• local statistical cues

Innovation

• syntactic distance
• visual self-referential instances
• global semantics
• phase transition
• vision models