The Deleuzian Representation Hypothesis

2025-12-17

AI Summary

This work addresses the problem that interpretable concept extraction in neural networks heavily relies on sparse autoencoders (SAEs) and lacks theoretical grounding. We propose an unsupervised differential clustering method that formalizes Deleuze’s philosophical principle—“concepts are differences”—into a computational framework: neuron activation differences are modeled via discriminant analysis, and activation skewness weighting is introduced to enhance concept diversity. The method is cross-modal, applicable uniformly to vision, language, and audio tasks. Evaluated on five mainstream models, it yields concepts of significantly higher quality than existing unsupervised SAE variants—approaching supervised baselines—while enabling causal intervention and effective behavioral control of neural models.

Technology Category

• Machine Learning: Unsupervised & Self-Supervised Learning
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Computer Vision: Interpretability, Explainability, and Transparency

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

We propose an alternative to sparse autoencoders (SAEs) as a simple and effective unsupervised method for extracting interpretable concepts from neural networks. The core idea is to cluster differences in activations, which we formally justify within a discriminant analysis framework. To enhance the diversity of extracted concepts, we refine the approach by weighting the clustering using the skewness of activations. The method aligns with Deleuze's modern view of concepts as differences. We evaluate the approach across five models and three modalities (vision, language, and audio), measuring concept quality, diversity, and consistency. Our results show that the proposed method achieves concept quality surpassing prior unsupervised SAE variants while approaching supervised baselines, and that the extracted concepts enable steering of a model's inner representations, demonstrating their causal influence on downstream behavior.

Problem

• Extracts interpretable concepts from neural networks unsupervised.
• Clusters activation differences to enhance concept diversity.
• Enables model steering through causal influence on behavior.

Innovation

• Clustering activation differences for concept extraction
• Weighting clustering with activation skewness for diversity
• Evaluating across models and modalities for quality