Nearest-Neighbor Density Estimation for Dependency Suppression

2026-03-04

AI Summary

This work proposes a novel encoder-based approach to eliminate unnecessary dependencies between data representations and sensitive variables while preserving data utility. Departing from conventional decorrelation or adversarial learning strategies, the method explicitly models and optimizes statistical independence by integrating non-parametric nearest-neighbor density estimation into a variational autoencoder framework. A custom-designed loss function directly suppresses dependence without relying on supervised signals. Empirical evaluations across multiple datasets demonstrate that the proposed method significantly outperforms existing unsupervised techniques, achieving an exceptional balance between removing sensitive information and retaining utility—performance that even rivals that of supervised approaches.

Technology Category

• Machine Learning: Unsupervised & Self-Supervised Learning
• Reasoning under Uncertainty: Graphical Models
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Security and Privacy: Data transparency and provenance
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

The ability to remove unwanted dependencies from data is crucial in various domains, including fairness, robust learning, and privacy protection. In this work, we propose an encoder-based approach that learns a representation independent of a sensitive variable but otherwise preserving essential data characteristics. Unlike existing methods that rely on decorrelation or adversarial learning, our approach explicitly estimates and modifies the data distribution to neutralize statistical dependencies. To achieve this, we combine a specialized variational autoencoder with a novel loss function driven by non-parametric nearest-neighbor density estimation, enabling direct optimization of independence. We evaluate our approach on multiple datasets, demonstrating that it can outperform existing unsupervised techniques and even rival supervised methods in balancing information removal and utility.

Problem

• dependency suppression
• nearest-neighbor density estimation
• independence
• sensitive variable
• data representation

Innovation

• nearest-neighbor density estimation
• dependency suppression
• variational autoencoder
• statistical independence
• unsupervised representation learning