Capsule Network Projectors are Equivariant and Invariant Learners

2024-05-23

arXiv.org

AI Summary

This work addresses the challenge of jointly modeling invariance and equivariance in self-supervised learning. We propose CapsIE—the first architecture to systematically integrate capsule networks (CapsNets) into equivariant self-supervised learning. Its core innovations are: (1) leveraging CapsNets’ intrinsic pose vectors to explicitly encode both class-level invariance and geometric equivariance under transformations; and (2) introducing an entropy-minimization-based joint objective for parameter-efficient, architecture-agnostic co-learning of invariant and equivariant representations. On the 3DIEBench rotation benchmark, CapsIE achieves state-of-the-art performance in equivariant self-supervised learning—matching fully supervised models—and demonstrates strong generalization across multi-task, large-scale scenarios.

Technology Category

• Machine Learning: Unsupervised & Self-Supervised Learning
• Computer Vision: Representation Learning for Vision
• Intelligent Robots: Learning & Optimization for ROB

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Learning invariant representations has been the longstanding approach to self-supervised learning. However, recently progress has been made in preserving equivariant properties in representations, yet do so with highly prescribed architectures. In this work, we propose an invariant-equivariant self-supervised architecture that employs Capsule Networks (CapsNets) which have been shown to capture equivariance with respect to novel viewpoints. We demonstrate that the use of CapsNets in equivariant self-supervised architectures achieves improved downstream performance on equivariant tasks with higher efficiency and fewer network parameters. To accommodate the architectural changes of CapsNets, we introduce a new objective function based on entropy minimisation. This approach which we name CapsIE (Capsule Invariant Equivariant Network) achieves state-of-the-art performance on the equivariant rotation tasks on the 3DIEBench dataset compared to prior equivariant SSL methods, while performing competitively against supervised counterparts. Our results demonstrate the ability of CapsNets to learn complex and generalised representations for large-scale, multi-task datasets compared to previous CapsNet benchmarks. Code is available at https://github.com/AberdeenML/CapsIE.

Problem

• Learning invariant and equivariant representations with Capsule Networks
• Improving efficiency and reducing parameters in equivariant self-supervised learning
• Achieving state-of-the-art performance on 3D rotation tasks

Innovation

• Uses Capsule Networks for equivariant self-supervised learning
• Introduces entropy minimization as new objective function
• Achieves efficient performance with fewer network parameters