Variational Learning of Disentangled Representations

2025-06-20

AI Summary

In multi-condition representation learning, shared and condition-specific latent factors are often entangled, impairing model generalization across treatments, patients, or species. To address this, we propose DISCoVeR—a novel unsupervised framework featuring a dual-latent-variable architecture and an adversarial max-min optimization objective that enforces strict disentanglement without requiring manual priors. Theoretically, we prove the uniqueness of the equilibrium solution and its consistency with maximum-likelihood estimation. Methodologically, DISCoVeR integrates variational autoencoding, dual-path reconstruction, and information bottleneck regularization. Empirically, it achieves significant improvements in disentanglement metrics and cross-condition generalization on synthetic data, natural images, and single-cell RNA-seq datasets—demonstrating robust extraction of biologically stable signals.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Search and Optimization: Mixed Discrete/Continuous Search
• Computer Vision: Representation Learning for Vision

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Disentangled representations enable models to separate factors of variation that are shared across experimental conditions from those that are condition-specific. This separation is essential in domains such as biomedical data analysis, where generalization to new treatments, patients, or species depends on isolating stable biological signals from context-dependent effects. While extensions of the variational autoencoder (VAE) framework have been proposed to address this problem, they frequently suffer from leakage between latent representations, limiting their ability to generalize to unseen conditions. Here, we introduce DISCoVeR, a new variational framework that explicitly separates condition-invariant and condition-specific factors. DISCoVeR integrates three key components: (i) a dual-latent architecture that models shared and specific factors separately; (ii) two parallel reconstructions that ensure both representations remain informative; and (iii) a novel max-min objective that encourages clean separation without relying on handcrafted priors, while making only minimal assumptions. Theoretically, we show that this objective maximizes data likelihood while promoting disentanglement, and that it admits a unique equilibrium. Empirically, we demonstrate that DISCoVeR achieves improved disentanglement on synthetic datasets, natural images, and single-cell RNA-seq data. Together, these results establish DISCoVeR as a principled approach for learning disentangled representations in multi-condition settings.

Problem

• Separate shared and condition-specific factors in data
• Prevent leakage between latent representations in VAEs
• Improve generalization to unseen experimental conditions

Innovation

• Dual-latent architecture separates shared and specific factors
• Parallel reconstructions maintain informative representations
• Max-min objective ensures clean separation without priors