Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Polytopal Neural Networks (PNNs), a novel interpretability framework that enforces a polytope-based structure within latent representations to capture layer-wise aspects directly during information processing. By leveraging learned corpus representations and amortized simplex inference, PNNs enable explicit observation descriptions via alignment with these learned aspects, offering a principled path to transparent AI. Empirical results demonstrate that PNNs preserve latent space structure with minimal performance degradation, yield favorable compressed representations over VQ, and provide a performant VQ training approach.
Neural networks can learn interpretable, polytope-based latent representations that preserve performance and enable direct VQ training.
Understanding how deep neural networks process information remains a central challenge. Existing interpretability methods often compromise structural fidelity, rely on prespecified corpora, or explain models post-hoc. We propose Polytopal Neural Networks (PNNs), a framework that extracts distinct layer-wise aspects by enforcing a polytope-based structure that is used directly in subsequent information processing. We scale our approach using learned corpus representations and an amortized simplex inference procedure and highlight how the framework also gives a direct route to vector quantized (VQ) training. In PNNs, observations are explicitly described by their alignment with layer-specific aspects. Empirical results show that imposing polytopal constraints on neural network representations preserves meaningful structures in the latent space with minimal degradation in performance, favorable compressed representations when compared to VQ representations in unsupervised learning, while also providing a performant new approach to VQ deep learning training. Our findings suggest that deep networks can enforce interpretable polytope-based representations, offering a principled path toward more transparent AI systems with minimal performance compromise.