Superposed Latent Autoencoder

2026-09-01

AI Summary

本文提出Superposed Latent Autoencoder，通过学习叠加多个宽潜变量共享存储，解决了传统自编码器因潜变量尺寸减小导致的表征能力下降问题。

Abstract

Autoencoders typically meet tight latent-memory budgets by making each latent representation smaller, sacrificing representational capacity. We ask a different question: can multiple wider latents be stored together instead? We introduce the Superposed Latent Autoencoder (SLAE), which preserves high-capacity latent representations while sharing storage through learned superposition. SLAE transforms latents into storage-friendly codes, binds them with randomized keys, superposes multiple codes into a single memory tensor, and learns to recover each latent before decoding. Under the same storage budget, SLAE replaces irreversible dimensional bottlenecks with structured interference that can be suppressed. Across CIFAR-10/100, SVHN, STL-10, Tiny ImageNet, and a wide range of memory budgets, SLAE substantially improves the reconstruction--memory tradeoff, reducing reconstruction error by up to 56% over conventional autoencoders at matched storage. Further analysis shows that SLAE's advantage comes from making wider representations usable under the same storage budget. These gains also extend beyond reconstruction: the information preserved by SLAE improves downstream classification by up to 16.79 percentage points under the same memory budget. Our results suggest a new principle for representation compression: instead of making every latent smaller, keep representations wide and let them share memory.

Problem

• latent representation
• storage budget
• autoencoder
• superposition
• memory sharing

Innovation

• Superposed Latent Autoencoder
• Latent Representation
• Storage Budget
• Reconstruction Error
• Memory Sharing