Unsupervised Disentanglement of Content and Style via Variance-Invariance Constraints

2024-07-04

AI Summary

This work addresses unsupervised disentanglement of content and style representations in sequential data. We propose V3, the first method to formalize variance-invariance inductive bias as a learnable constraint: content encodings must be stable across samples yet variable within each sample, while style encodings exhibit the opposite behavior. Our encoder-decoder architecture integrates unsupervised contrastive regularization, a discrete content codebook, and semantic alignment to achieve cross-modal disentanglement, few-shot out-of-distribution generalization, and symbol-level interpretability. Evaluated across diverse domains—including music (pitch/timbre), images (digit identity/color), and animation (motion/character appearance)—V3 achieves state-of-the-art performance. Under few-shot adaptation, it surpasses supervised baselines; moreover, its learned content codebook exhibits strong alignment with human prior knowledge.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Computer Vision: Visual Reasoning & Symbolic Representations
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics

Abstract

We contribute an unsupervised method that effectively learns disentangled content and style representations from sequences of observations. Unlike most disentanglement algorithms that rely on domain-specific labels or knowledge, our method is based on the insight of domain-general statistical differences between content and style -- content varies more among different fragments within a sample but maintains an invariant vocabulary across data samples, whereas style remains relatively invariant within a sample but exhibits more significant variation across different samples. We integrate such inductive bias into an encoder-decoder architecture and name our method after V3 (variance-versus-invariance). Experimental results show that V3 generalizes across multiple domains and modalities, successfully learning disentangled content and style representations, such as pitch and timbre from music audio, digit and color from images of hand-written digits, and action and character appearance from simple animations. V3 demonstrates strong disentanglement performance compared to existing unsupervised methods, along with superior out-of-distribution generalization under few-shot adaptation compared to supervised counterparts. Lastly, symbolic-level interpretability emerges in the learned content codebook, forging a near one-to-one alignment between machine representation and human knowledge.

Problem

• Unsupervised disentanglement of content and style representations.
• Generalizes across multiple domains and modalities effectively.
• Achieves superior out-of-distribution generalization and interpretability.

Innovation

• Unsupervised disentanglement via variance-invariance constraints
• Encoder-decoder architecture with V3 method
• Generalizes across domains, modalities, few-shot adaptation