Make it SING: Analyzing Semantic Invariants in Classifiers

2026-03-15

AI Summary

This work addresses the challenge of interpreting semantically ambiguous invariants residing in the null space of existing image classifiers, which often defy human-understandable explanations. To this end, the authors propose SING, a novel method that, for the first time, leverages multimodal vision-language models for null space analysis. By constructing network-equivalent images and mapping classifier features into a vision-language embedding space, SING generates natural language descriptions alongside visual examples to provide semantic interpretations of equivalent input perturbations. The approach supports both single-image local analysis and multi-image statistical evaluation, revealing that ResNet50 leaks semantic attributes through its null space, whereas self-supervised pretrained DinoViT better preserves class-semantic consistency.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Security and Privacy: Data transparency and provenance

Abstract

All classifiers, including state-of-the-art vision models, possess invariants, partially rooted in the geometry of their linear mappings. These invariants, which reside in the null-space of the classifier, induce equivalent sets of inputs that map to identical outputs. The semantic content of these invariants remains vague, as existing approaches struggle to provide human-interpretable information. To address this gap, we present Semantic Interpretation of the Null-space Geometry (SING), a method that constructs equivalent images, with respect to the network, and assigns semantic interpretations to the available variations. We use a mapping from network features to multi-modal vision language models. This allows us to obtain natural language descriptions and visual examples of the induced semantic shifts. SING can be applied to a single image, uncovering local invariants, or to sets of images, allowing a breadth of statistical analysis at the class and model levels. For example, our method reveals that ResNet50 leaks relevant semantic attributes to the null space, whereas DinoViT, a ViT pretrained with self-supervised DINO, is superior in maintaining class semantics across the invariant space.

Problem

• semantic invariants
• null-space
• classifier interpretability
• vision models
• semantic content

Innovation

• null-space geometry
• semantic invariants
• vision-language models
• interpretable AI
• classifier analysis