Machine Unlearning in Hyperbolic vs. Euclidean Multimodal Contrastive Learning: Adapting Alignment Calibration to MERU

2025-03-19

AI Summary

This work pioneers the extension of machine unlearning to hyperbolic contrastive learning, specifically targeting interpretable removal of semantic concepts in multimodal pretrained models (MERU). Addressing the limitation of existing unlearning methods—confined to Euclidean geometry—we propose a hyperbolic alignment calibration framework featuring two novel components: entailment calibration and hyperbolic norm regularization. Unlike prior approaches relying on cross-modal disentanglement, our method achieves cooperative unlearning by reconstructing hierarchical semantic structures directly within hyperbolic space. Experiments demonstrate near-perfect unlearning on both single- and multi-concept removal tasks; crucially, retained performance on non-target concepts significantly surpasses Euclidean baselines—especially under complex multi-concept scenarios. Our approach establishes a new paradigm for controllable unlearning in hyperbolic representation spaces.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

Machine unlearning methods have become increasingly important for selective concept removal in large pre-trained models. While recent work has explored unlearning in Euclidean contrastive vision-language models, the effectiveness of concept removal in hyperbolic spaces remains unexplored. This paper investigates machine unlearning in hyperbolic contrastive learning by adapting Alignment Calibration to MERU, a model that embeds images and text in hyperbolic space to better capture semantic hierarchies. Through systematic experiments and ablation studies, we demonstrate that hyperbolic geometry offers distinct advantages for concept removal, achieving near perfect forgetting with reasonable performance on retained concepts, particularly when scaling to multiple concept removal. Our approach introduces hyperbolic-specific components including entailment calibration and norm regularization that leverage the unique properties of hyperbolic space. Comparative analysis with Euclidean models reveals fundamental differences in unlearning dynamics, with hyperbolic unlearning reorganizing the semantic hierarchy while Euclidean approaches merely disconnect cross-modal associations. These findings not only advance machine unlearning techniques but also provide insights into the geometric properties that influence concept representation and removal in multimodal models. Source code available at https://github.com/alex-pv01/HAC

Problem

• Explores machine unlearning in hyperbolic contrastive learning.
• Adapts Alignment Calibration to MERU for concept removal.
• Compares hyperbolic vs. Euclidean unlearning dynamics and effectiveness.

Innovation

• Adapts Alignment Calibration to MERU model
• Introduces hyperbolic-specific components for unlearning
• Demonstrates hyperbolic geometry advantages in concept removal