A Unified Framework for Vision Transformers Equivariant to Discrete Subgroups of $\mathrm{O}(2)$

2026-06-26

AI Summary

Standard Vision Transformers lack explicit modeling of common discrete planar symmetries in images—such as rotations and reflections—limiting their representational capacity in scenarios with symmetric structures. This work proposes the first unified framework that endows Vision Transformers with equivariance to arbitrary discrete subgroups of $\mathrm{O}(2)$, including $D_4$ and $D_6$, by introducing group-equivariant self-attention and homogeneous-space nonlinear activations for symmetry-aware representation learning. Theoretically, the authors establish embedding properties under group inclusion and prove the universal expressive power of single-head equivariant self-attention. They instantiate the framework with a $D_6$-equivariant architecture based on hexagonal image patches. Experiments on PatternNet under low-data regimes demonstrate significant performance gains over baselines at comparable parameter counts, validating the efficacy of explicit symmetry modeling.

Abstract

Vision transformers have become a dominant architecture for visual recognition. However, standard models do not explicitly encode the planar symmetries that arise in many vision domains. We introduce a family of vision transformers equivariant to arbitrary discrete subgroups of $\mathrm{O}(2)$, providing a unified framework that generalizes prior flipping- and $D_4$-equivariant transformer architectures. Our construction yields equivariant analogues of the core transformer components, together with expressivity guarantees for the resulting layers. In particular, we show that whenever $H \le G$, the class of $G$-equivariant ViTs embeds naturally into the class of $H$-equivariant ViTs. We also prove that, in the single-head setting, the corresponding equivariant self-attention layer realizes every $G$-equivariant self-attention map representable by ordinary self-attention. We further construct a $D_6$-equivariant model based on hexagonal patches, making the architecture compatible with six-fold rotational symmetries. We evaluate the resulting models on the PatternNet aerial image dataset in artificially data-scarce regimes across subgroups of $D_4$ and $D_6$. Our experiments compare two equivariant attention mechanisms and analyze how the choice of homogeneous-space configurations used in the nonlinearities affects performance. Preliminary results under matched parameter budgets indicate that equivariance can improve recognition accuracy, motivating further study of how discrete symmetry groups shape transformer-based visual recognition models.

Problem

• Vision Transformers
• Equivariance
• Discrete Symmetry Groups
• O(2)
• Visual Recognition

Innovation

• equivariant vision transformers
• discrete symmetry groups
• O(2) subgroups
• hexagonal patches
• self-attention equivariance