Equivariant Neural Networks for General Linear Symmetries on Lie Algebras

2025-10-27

AI Summary

Existing equivariant neural networks are largely restricted to compact groups (e.g., SO(3)) or vector-valued inputs, rendering them inadequate for modeling the generalized linear group GL(n) symmetries ubiquitous in scientific domains. To address this, we propose Reductive Lie Neurons (ReLNs)—the first unified architecture explicitly supporting *non-compact* GL(n)-equivariance. Grounded in Lie algebra representation theory, ReLNs introduce adjoint-invariant bilinear layers, equivariant activation functions, and matrix spectral embeddings, enabling end-to-end GL(n)-equivariant learning over n×n matrix inputs without subgroup-specific architectural redesign. On sl(3) and sp(4) Lie algebra tasks, ReLNs significantly outperform existing baselines. Moreover, they demonstrate strong generalization and improved accuracy in Lorentz-equivariant particle physics modeling and 3D drone state estimation—two challenging real-world applications demanding non-compact group equivariance.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Natural Language Processing: Safety and Robustness

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

Encoding symmetries is a powerful inductive bias for improving the generalization of deep neural networks. However, most existing equivariant models are limited to simple symmetries like rotations, failing to address the broader class of general linear transformations, GL(n), that appear in many scientific domains. We introduce Reductive Lie Neurons (ReLNs), a novel neural network architecture exactly equivariant to these general linear symmetries. ReLNs are designed to operate directly on a wide range of structured inputs, including general n-by-n matrices. ReLNs introduce a novel adjoint-invariant bilinear layer to achieve stable equivariance for both Lie-algebraic features and matrix-valued inputs, without requiring redesign for each subgroup. This architecture overcomes the limitations of prior equivariant networks that only apply to compact groups or simple vector data. We validate ReLNs' versatility across a spectrum of tasks: they outperform existing methods on algebraic benchmarks with sl(3) and sp(4) symmetries and achieve competitive results on a Lorentz-equivariant particle physics task. In 3D drone state estimation with geometric uncertainty, ReLNs jointly process velocities and covariances, yielding significant improvements in trajectory accuracy. ReLNs provide a practical and general framework for learning with broad linear group symmetries on Lie algebras and matrix-valued data. Project page: https://reductive-lie-neuron.github.io/

Problem

• Achieving equivariance to general linear symmetries GL(n)
• Overcoming limitations of prior networks for compact groups
• Enabling stable equivariance for Lie-algebraic and matrix-valued data

Innovation

• ReLNs achieve equivariance to general linear symmetries
• Novel adjoint-invariant bilinear layer ensures stable equivariance
• Architecture processes Lie-algebraic features and matrix-valued inputs