Interpret Your Decision: Logical Reasoning Regularization for Generalization in Visual Classification

2024-10-06

arXiv.org

AI Summary

Vision classification models suffer from poor generalization and lack of interpretability under unknown-class and unknown-domain settings. To address this, we propose Logic-regularized learning (L-Reg), the first framework to establish a theoretical connection between formal logic and visual generalization: it incorporates semantic part-based logical rules (e.g., “face → person”) into the training objective, jointly constraining feature distribution complexity and classifier weight norms to achieve interpretable and robust discrimination. Our method supports both multi-domain joint training and generalized category discovery. We provide theoretical analysis proving that L-Reg effectively reduces model complexity. Empirically, L-Reg achieves significant accuracy gains on multi-domain generalization and generalized category discovery benchmarks; notably, it delivers consistently improved generalization under unknown-class and unknown-domain conditions, simultaneously attaining strong generalization capability and intrinsic interpretability.

Technology Category

• Computer Vision: Visual Reasoning & Symbolic Representations
• Machine Learning: Statistical Relational/Logic Learning
• Knowledge Representation and Reasoning: Logic Programming

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Vision models excel in image classification but struggle to generalize to unseen data, such as classifying images from unseen domains or discovering novel categories. In this paper, we explore the relationship between logical reasoning and deep learning generalization in visual classification. A logical regularization termed L-Reg is derived which bridges a logical analysis framework to image classification. Our work reveals that L-Reg reduces the complexity of the model in terms of the feature distribution and classifier weights. Specifically, we unveil the interpretability brought by L-Reg, as it enables the model to extract the salient features, such as faces to persons, for classification. Theoretical analysis and experiments demonstrate that L-Reg enhances generalization across various scenarios, including multi-domain generalization and generalized category discovery. In complex real-world scenarios where images span unknown classes and unseen domains, L-Reg consistently improves generalization, highlighting its practical efficacy.

Problem

• Visual Model
• Adaptability
• Unseen Categories

Innovation

• L-Reg
• Logic Reasoning
• Domain Adaptation