From MNIST to ImageNet: Understanding the Scalability Boundaries of Differentiable Logic Gate Networks

2025-09-30

AI Summary

This work investigates the scalability and expressive power of Differentiable Logic Gate Networks (DLGNs) for large-scale, multi-class visual recognition tasks. Addressing high-cardinality classification (up to 2,000 classes), we introduce a temperature-tuning mechanism to stabilize output distributions and systematically evaluate multiple output strategies, identifying the practical applicability boundary of the Group-Sum layer for thousand-class settings. Extensive experiments on both synthetic and real-world image datasets empirically validate, for the first time, the feasibility and robustness of DLGNs under ultra-large-scale classification. Key contributions are: (1) a combined theoretical and empirical demonstration of DLGNs’ universal approximation capability; (2) identification of the critical role of the temperature parameter in regulating logic gate output distributions; and (3) establishment of the first scalability benchmark and architectural design guidelines for extending DLGNs to ImageNet-scale (10,000-class) tasks.

Technology Category

• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Computer Vision: Large Vision Models
• Knowledge Representation and Reasoning: Logic Programming

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies

Abstract

Differentiable Logic Gate Networks (DLGNs) are a very fast and energy-efficient alternative to conventional feed-forward networks. With learnable combinations of logical gates, DLGNs enable fast inference by hardware-friendly execution. Since the concept of DLGNs has only recently gained attention, these networks are still in their developmental infancy, including the design and scalability of their output layer. To date, this architecture has primarily been tested on datasets with up to ten classes. This work examines the behavior of DLGNs on large multi-class datasets. We investigate its general expressiveness, its scalability, and evaluate alternative output strategies. Using both synthetic and real-world datasets, we provide key insights into the importance of temperature tuning and its impact on output layer performance. We evaluate conditions under which the Group-Sum layer performs well and how it can be applied to large-scale classification of up to 2000 classes.

Problem

• Investigating DLGN scalability on large multi-class datasets
• Evaluating alternative output strategies for DLGN architectures
• Assessing temperature tuning impact on output layer performance

Innovation

• Uses differentiable logic gate networks for efficiency
• Explores scalability with Group-Sum layer for classification
• Employs temperature tuning to optimize output performance