Spatial Colour Mixing Illusions as a Perception Stress Test for Vision-Language Models

2026-03-06

AI Summary

This work proposes a novel perceptual stress test based on spatial color-mixing illusions to probe the robustness of vision-language models under structured chromatic perturbations. The authors generate eight procedural color distortion variants by combining RGB and Ostwald color systems to produce structured interference patterns. Systematic evaluation across nine prominent models and multiple datasets reveals significant performance degradation that is not easily mitigated by scaling. In contrast, human observers maintain robust recognition under the same conditions. Inspired by human visual mechanisms, a simple preprocessing strategy partially restores model robustness. These findings are further corroborated by psychophysical experiments involving 61 human participants, highlighting a pronounced gap between human and machine perception in handling structured color perturbations.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Vision-language models (VLMs) achieve strong benchmark results, yet can exhibit systematic perceptual weaknesses: structured, large changes to pixel values can cause confident yet nonsensical predictions, even when the underlying scene remains easily recognizable to humans. We study this gap using Spatial Colour Mixing, a programmatic family of colour distortions that overlays structured patterns (in both RGB and Ostwald colour systems) onto natural images. We introduce a framework of eight spatial colour mixing variants and evaluate nine VLMs across three model families on four datasets. Across models and datasets, accuracy degrades sharply with increasing distortion, and scaling the language model does not reliably mitigate the failure. In a human study with 61 participants on an animal recognition dataset, humans substantially outperform VLMs under the same distortions. Finally, we show that a simple human-inspired preprocessing step recovers a meaningful portion of performance for several distortion types, motivating perception-aware preprocessing and tool-use as practical strategies for improving VLM robustness.

Problem

• vision-language models
• spatial colour mixing
• perceptual robustness
• colour distortion
• model failure

Innovation

• Spatial Colour Mixing
• Vision-Language Models
• Perceptual Robustness
• Colour Distortion
• Human-Inspired Preprocessing