Search papers, labs, and topics across Lattice.
This paper introduces Normalizing Autoencoder (NAE), a novel framework that optimizes the training dynamics of flow autoencoders by aligning the surrogate loss gradient with the reconstruction loss. The authors prove that existing loss functions are suboptimal and demonstrate through extensive experiments that NAE achieves state-of-the-art performance across various domains, including molecule generation and image benchmarks. This work underscores the critical role of loss alignment in enhancing the efficacy of generative models.
Aligning surrogate and reconstruction losses in flow autoencoders leads to state-of-the-art generative performance across diverse applications.
We consider the setting of Normalizing flows with approximate inverses, an established paradigm spanning both full-dimensional ($d=D$) and bottleneck ($d<D$) settings, and group these models under the term flow autoencoders. We present a theoretical investigation into their training dynamics and prove that the proposed loss used by existing approaches is suboptimal; specifically, both encoder and decoder surrogates must be optimized in alignment with reconstruction loss. Guided by these insights, we propose Normalizing Autoencoder (NAE), which employs a novel conditional loss that aligns the surrogate loss gradient with that of reconstruction loss, directly improving upon the current standard. Extensive experiments across molecule generation, tabular data, and image benchmarks demonstrate that NAE achieves state of the art performance. Our work highlights the importance of loss alignment in flow autoencoders and establishes NAE as a powerful generative framework.
