Staying on the Manifold: Geometry-Aware Noise Injection

2025-09-24

AI Summary

This work addresses the misalignment between input noise and data geometric structure in deep learning. We propose a geometry-aware input noise injection method: Gaussian noise is first projected onto the tangent space of the data manifold and then mapped back onto the manifold along geodesics; alternatively, Brownian motion is directly simulated on the manifold. Our approach explicitly leverages differential-geometric properties—such as curvature and geodesic distance—either from an explicitly modeled or learned manifold, ensuring injected noise better conforms to the intrinsic data distribution. Experiments demonstrate that the method significantly improves model generalization and hyperparameter robustness on high-curvature manifolds, while matching or exceeding the performance of noise-free training on simpler manifolds—validating its effectiveness and broad applicability. The core contribution is the first systematic integration of manifold geometric priors into input perturbation design, thereby bridging differential geometry and robust deep learning.

Technology Category

• Machine Learning: Learning with Manifolds
• Computer Vision: Adversarial Attacks & Robustness
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

It has been shown that perturbing the input during training implicitly regularises the gradient of the learnt function, leading to smoother models and enhancing generalisation. However, previous research mostly considered the addition of ambient noise in the input space, without considering the underlying structure of the data. In this work, we propose several methods of adding geometry-aware input noise that accounts for the lower dimensional manifold the input space inhabits. We start by projecting ambient Gaussian noise onto the tangent space of the manifold. In a second step, the noise sample is mapped on the manifold via the associated geodesic curve. We also consider Brownian motion noise, which moves in random steps along the manifold. We show that geometry-aware noise leads to improved generalization and robustness to hyperparameter selection on highly curved manifolds, while performing at least as well as training without noise on simpler manifolds. Our proposed framework extends to learned data manifolds.

Problem

• Developing geometry-aware noise injection methods for data manifolds
• Improving model generalization through manifold-structured noise perturbation
• Enhancing robustness to hyperparameter selection on curved manifolds

Innovation

• Projecting Gaussian noise onto manifold tangent spaces
• Mapping noise samples via geodesic curves on manifolds
• Using Brownian motion noise along manifold structures