Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces the Superposed Latent Autoencoder (SLAE), which innovatively allows multiple wider latent representations to be stored together, enhancing representational capacity without exceeding tight memory budgets. By transforming latents into storage-friendly codes and employing learned superposition, SLAE achieves significant improvements in the reconstruction-memory tradeoff, reducing reconstruction error by up to 56% compared to traditional autoencoders. Additionally, SLAE enhances downstream classification performance by up to 16.79 percentage points, demonstrating that wider representations can be effectively utilized within the same storage constraints.
Storing wider latent representations together can slash reconstruction errors by over half while boosting classification accuracy, challenging conventional wisdom on latent size constraints.
Autoencoders typically meet tight latent-memory budgets by making each latent representation smaller, sacrificing representational capacity. We ask a different question: can multiple wider latents be stored together instead? We introduce the Superposed Latent Autoencoder (SLAE), which preserves high-capacity latent representations while sharing storage through learned superposition. SLAE transforms latents into storage-friendly codes, binds them with randomized keys, superposes multiple codes into a single memory tensor, and learns to recover each latent before decoding. Under the same storage budget, SLAE replaces irreversible dimensional bottlenecks with structured interference that can be suppressed. Across CIFAR-10/100, SVHN, STL-10, Tiny ImageNet, and a wide range of memory budgets, SLAE substantially improves the reconstruction--memory tradeoff, reducing reconstruction error by up to 56% over conventional autoencoders at matched storage. Further analysis shows that SLAE's advantage comes from making wider representations usable under the same storage budget. These gains also extend beyond reconstruction: the information preserved by SLAE improves downstream classification by up to 16.79 percentage points under the same memory budget. Our results suggest a new principle for representation compression: instead of making every latent smaller, keep representations wide and let them share memory.