🤖 AI Summary
In disentangled representation learning, statistical independence does not guarantee semantic irrelevance, rendering conventional independence-based methods incapable of ensuring semantic disentanglement. This work identifies this fundamental inconsistency and proposes a novel “difference-driven disentanglement” paradigm: it abandons the latent-variable independence assumption and instead explicitly models the intrinsic semantic distinctions among factors. Specifically, we design a difference encoder to capture discriminative semantic features across factors and introduce a cross-dimensional contrastive loss to achieve explicit, semantic-level disentanglement in a fully unsupervised manner. Evaluated on dSprites and 3DShapes, our method consistently outperforms state-of-the-art disentanglement models across multiple standard metrics—including DCI, SAP, and MIG—demonstrating both the effectiveness and generalizability of semantic difference modeling for disentanglement.
📝 Abstract
In this study, Disentanglement in Difference(DiD) is proposed to address the inherent inconsistency between the statistical independence of latent variables and the goal of semantic disentanglement in disentanglement representation learning. Conventional disentanglement methods achieve disentanglement representation by improving statistical independence among latent variables. However, the statistical independence of latent variables does not necessarily imply that they are semantically unrelated, thus, improving statistical independence does not always enhance disentanglement performance. To address the above issue, DiD is proposed to directly learn semantic differences rather than the statistical independence of latent variables. In the DiD, a Difference Encoder is designed to measure the semantic differences; a contrastive loss function is established to facilitate inter-dimensional comparison. Both of them allow the model to directly differentiate and disentangle distinct semantic factors, thereby resolving the inconsistency between statistical independence and semantic disentanglement. Experimental results on the dSprites and 3DShapes datasets demonstrate that the proposed DiD outperforms existing mainstream methods across various disentanglement metrics.