Rank-Aware Hyperbolic Alignment for Vision-Language Dataset Distillation

2026-06-28

AI Summary

Existing vision-language distillation methods enforce full-dimensional alignment in Euclidean space, overlooking the intrinsically low-rank nature of cross-modal correlations, which leads to over-constrained representations and limited generalization. This work introduces hyperbolic geometry into multimodal distillation for the first time, proposing a low-rank-aware alignment framework. By employing an asymmetric distillation objective, it achieves geodesic alignment within a shared semantic subspace in hyperbolic space while regularizing the residual subspace to preserve modality-specific information. This approach explicitly controls alignment capacity, circumventing the limitations of Euclidean geometry and significantly enhancing cross-modal retrieval performance and robustness in downstream task transfer under fixed data and computational budgets.

Abstract

Vision-language dataset distillation (VLDD) compresses a large image-text paired dataset into a small set of synthetic pairs that can efficiently train contrastive vision-language models under strict data and compute budgets. Most existing methods match expert trajectories or cross-modal statistics, yet still enforce full-dimensional alignment in a Euclidean embedding space. This is often overly restrictive due to rank-deficient image--text correlation, with shared semantics concentrated in a low-dimensional range and remaining variation spread across a weakly correlated residual subspace. LoRS relaxes alignment at the similarity level by low-rank factorization, but does not explicitly control dominant alignment capacity and structure in the representation space. We thus propose a rank-aware hyperbolic alignment (RAHA) that combines hierarchical geometry with explicit alignment-capacity control. RAHA lifts multimodal representations to hyperbolic space and optimizes distilled pairs with asymmetric objectives that enforce geodesic alignment in the shared range while regularizing the residual subspace to preserve modality-private diversity and improve transfer robustness. Experiments on benchmarks show that RAHA demonstrates competitive cross-modal retrieval and improved transfer indicators under fixed budgets.

Problem

• vision-language dataset distillation
• rank-deficient correlation
• embedding alignment
• hyperbolic space
• modality-private diversity

Innovation

• rank-aware
• hyperbolic alignment
• vision-language distillation
• low-rank correlation
• geodesic alignment