Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces ODIN, an innovative autoencoder architecture designed to recover PCA-like latent structures in a non-linear setting by enforcing geometric constraints during training. By ensuring that latent dimensions remain orthogonal and ordered by explained variance, ODIN achieves a balance between interpretability and the expressive capabilities of deep networks. Empirical evaluations on both synthetic and real-world datasets validate its effectiveness in structured feature learning and dimensionality reduction.
ODIN achieves PCA-like interpretability in deep autoencoders without sacrificing non-linear expressiveness, paving the way for more structured feature learning.
Principal Component Analysis or PCA-like properties (orthogonality, variance ranking) are seldom realized in deep autoencoder architectures. In this work, we present ODIN (Orthogonal Dendritic Intrinsic Network), a novel autoencoder architecture that recovers PCA-like latent structure in a fully non-linear regime. By incorporating a set of geometric constraints directly into the training objective, ODIN encourages latent dimensions to be mutually orthogonal and ordered by explained variance, mirroring the interpretable decomposition of PCA while retaining the expressive power of deep networks. We provide theoretical grounding for these constraints and demonstrate their compatibility with standard encoder-decoder frameworks. We also establish empirical results for both synthetic and real world datasets, establishing a principled path toward interpretable, structured feature learning and dimensionality reduction.