Product Interaction: An Algebraic Formalism for Deep Learning Architectures

2026-01-31

AI Summary

This work addresses the absence of a unified algebraic framework for systematically characterizing the diverse layer structures in modern deep learning architectures. It proposes a product interaction formalism that generates algebraic expressions for neural network layers by defining multiplicative operators and symmetry constraints over appropriate algebraic structures, organized by interaction order. For the first time, this approach unifies major architectures—including convolutions, equivariant networks, attention mechanisms, and Mamba—within a cohesive algebraic framework based on linear and higher-order product interactions. The framework not only provides a unified algebraic representation of existing models but also establishes systematic design principles for constructing novel network architectures.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Computer Vision: Large Vision Models
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

In this paper, we introduce product interactions, an algebraic formalism in which neural network layers are constructed from compositions of a multiplication operator defined over suitable algebras. Product interactions provide a principled way to generate and organize algebraic expressions by increasing interaction order. Our central observation is that algebraic expressions in modern neural networks admit a unified construction in terms of linear, quadratic, and higher-order product interactions. Convolutional and equivariant networks arise as symmetry-constrained linear product interactions, while attention and Mamba correspond to higher-order product interactions.

Problem

• product interactions
• algebraic formalism
• deep learning architectures
• neural network layers
• interaction order

Innovation

• product interactions
• algebraic formalism
• neural network architecture
• higher-order interactions
• symmetry constraints