Orthogonal Dendritic Intrinsic Networks: An Architecture for Significance-Ordered, Orthogonal Latent Spaces

2026-07-06

AI Summary

Deep autoencoders typically struggle to produce latent representations that are orthogonal and ordered by explained variance, as in principal component analysis (PCA), which limits their interpretability. This work proposes the ODIN architecture, which incorporates geometric constraints into a standard encoder–decoder framework to enable, for the first time, end-to-end learning of orthogonal and variance-ordered latent dimensions in a fully nonlinear deep autoencoder. By doing so, ODIN preserves the expressive power of deep models while recovering the structural clarity and interpretability inherent to PCA. The method is validated on both synthetic and real-world datasets, demonstrating its effectiveness and establishing a new paradigm for interpretable nonlinear dimensionality reduction.

Abstract

Principal Component Analysis or PCA-like properties (orthogonality, variance ranking) are seldom realized in deep autoencoder architectures. In this work, we present ODIN (Orthogonal Dendritic Intrinsic Network), a novel autoencoder architecture that recovers PCA-like latent structure in a fully non-linear regime. By incorporating a set of geometric constraints directly into the training objective, ODIN encourages latent dimensions to be mutually orthogonal and ordered by explained variance, mirroring the interpretable decomposition of PCA while retaining the expressive power of deep networks. We provide theoretical grounding for these constraints and demonstrate their compatibility with standard encoder-decoder frameworks. We also establish empirical results for both synthetic and real world datasets, establishing a principled path toward interpretable, structured feature learning and dimensionality reduction.

Problem

• autoencoder
• orthogonality
• latent space
• PCA
• dimensionality reduction

Innovation

• orthogonal latent space
• autoencoder
• PCA-like representation
• structured feature learning
• variance-ordered encoding