Bidirectional Variational Autoencoders

2024-06-30

IEEE International Joint Conference on Neural Network

AI Summary

Traditional variational autoencoders (VAEs) suffer from parameter redundancy and disjoint encoder-decoder architectures. To address these issues, this paper proposes the Bidirectional VAE (BVAE), which employs a single shared-weight neural network to jointly realize forward encoding and reverse decoding, establishing an end-to-end differentiable bidirectional information flow. This design uniquely integrates variational inference with parameter sharing at the architectural level. Experiments on MNIST, Fashion-MNIST, CIFAR-10, and CelebA-64 demonstrate that BVAE reduces model parameters by approximately 50%, while simultaneously improving reconstruction fidelity, enhancing generative diversity, slightly boosting downstream classification accuracy, and significantly increasing interpolation smoothness. The core contribution lies in achieving efficient bidirectional modeling via a minimalist architecture—offering a novel paradigm for lightweight generative modeling.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Natural Language Processing: Generation

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

We present the new bidirectional variational autoencoder (BVAE) network architecture. The BVAE uses a single neural network both to encode and decode instead of an encoder-decoder network pair. The network encodes in the forward direction and decodes in the backward direction through the same synaptic web. Simulations compared BVAEs and ordinary VAEs on the four image tasks of image reconstruction, classification, interpolation, and generation. The image datasets included MNIST handwritten digits, Fashion-MNIST, CIFAR10, and CelebA-64 face images. The bidirectional structure of BVAEs cut the parameter count by almost 50% and still slightly outperformed the unidirectional VAEs.

Problem

• BVAE reduces parameters by 50% vs VAEs
• BVAE performs encoding and decoding bidirectionally
• BVAE outperforms VAEs in image tasks

Innovation

• Single network for encoding and decoding
• Bidirectional synaptic web structure
• Reduced parameters by 50% with better performance