PH-VAE: A Polynomial Hierarchical Variational Autoencoder Towards Disentangled Representation Learning

2025-02-05

AI Summary

To address key limitations of variational autoencoders (VAEs)—including poor interpretability of latent variables, blurry reconstructions, the “prior collapse” (or “origin attraction”) effect induced by KL divergence, and overfitting in low-data regimes—this paper proposes the Polynomial Hierarchical VAE (PH-VAE). Methodologically, PH-VAE introduces: (1) a novel polynomial divergence to replace KL divergence, thereby alleviating excessive prior constraint; (2) a hierarchical latent structure governed by polynomial transformations, enhancing representation disentanglement and robustness; and (3) explicit latent-space disentanglement regularization to improve fine-grained feature modeling. Experiments demonstrate that PH-VAE significantly outperforms baseline VAEs in reconstruction sharpness, distribution fidelity, and quantitatively verifiable disentanglement—particularly under small-sample settings, where it effectively mitigates origin attraction. Overall, PH-VAE establishes a new paradigm for interpretable generative modeling.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Computer Vision: Representation Learning for Vision
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

The variational autoencoder (VAE) is a simple and efficient generative artificial intelligence method for modeling complex probability distributions of various types of data, such as images and texts. However, it suffers some main shortcomings, such as lack of interpretability in the latent variables, difficulties in tuning hyperparameters while training, producing blurry, unrealistic downstream outputs or loss of information due to how it calculates loss functions and recovers data distributions, overfitting, and origin gravity effect for small data sets, among other issues. These and other limitations have caused unsatisfactory generation effects for the data with complex distributions. In this work, we proposed and developed a polynomial hierarchical variational autoencoder (PH-VAE), in which we used a polynomial hierarchical date format to generate or to reconstruct the data distributions. In doing so, we also proposed a novel Polynomial Divergence in the loss function to replace or generalize the Kullback-Leibler (KL) divergence, which results in systematic and drastic improvements in both accuracy and reproducibility of the re-constructed distribution function as well as the quality of re-constructed data images while keeping the dataset size the same but capturing fine resolution of the data. Moreover, we showed that the proposed PH-VAE has some form of disentangled representation learning ability.

Problem

• Improve interpretability in latent variables
• Enhance accuracy in reconstructed data
• Enable disentangled representation learning

Innovation

• Polynomial hierarchical data format
• Novel Polynomial Divergence
• Disentangled representation learning