Multi-Way Representation Alignment

2026-02-05

AI Summary

Existing representation alignment methods are largely confined to pairwise model alignment, limiting their scalability to multi-model settings and their ability to establish a consistent global reference space. This work proposes Geometrically Corrected Procrustes Alignment (GCPA), which leverages generalized Procrustes analysis to construct a shared orthogonal space while incorporating a posterior directional correction mechanism. This approach enables efficient and consistent multi-way alignment without compromising the intrinsic geometric structure of individual models. Experimental results demonstrate that GCPA consistently improves performance across cross-model retrieval tasks between any pair of models and maintains a practical, unified shared reference space.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

The Platonic Representation Hypothesis suggests that independently trained neural networks converge to increasingly similar latent spaces. However, current strategies for mapping these representations are inherently pairwise, scaling quadratically with the number of models and failing to yield a consistent global reference. In this paper, we study the alignment of $M \ge 3$ models. We first adapt Generalized Procrustes Analysis (GPA) to construct a shared orthogonal universe that preserves the internal geometry essential for tasks like model stitching. We then show that strict isometric alignment is suboptimal for retrieval, where agreement-maximizing methods like Canonical Correlation Analysis (CCA) typically prevail. To bridge this gap, we finally propose Geometry-Corrected Procrustes Alignment (GCPA), which establishes a robust GPA-based universe followed by a post-hoc correction for directional mismatch. Extensive experiments demonstrate that GCPA consistently improves any-to-any retrieval while retaining a practical shared reference space.

Problem

• representation alignment
• multi-way alignment
• shared reference space
• neural network representations
• Platonic Representation Hypothesis

Innovation

• Multi-way alignment
• Generalized Procrustes Analysis
• Geometry-Corrected Procrustes Alignment
• Latent space alignment
• Model stitching