Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces the Empirical Variational Autoencoder (EVA), a novel generative framework for continuous-valued sequences that empirically learns autoregressive latent priors from data via a single linear layer. By replacing fixed Gaussian priors with learned ones, EVA effectively bridges the gap between prior and posterior distributions, enabling high-fidelity ancestral sampling. Experimental results on image and sound synthesis show EVA achieves competitive generation quality with autoregressive diffusion models but with significantly faster inference.
Autoregressive latent priors learned directly from data, not fixed Gaussians, unlock VAEs for high-fidelity sequential generation with diffusion-level quality and faster inference.
We present Empirical Variational Autoencoder, a general generative framework for continuous-valued (i.e., non-vector-quantized) sequences. EVA is based on the evidence lower bound of the Variational Autoencoder (VAE) but learns autoregressive latent priors empirically from training data, which can be implemented only by an additional single linear layer on top of VAEs. By replacing the conventional standard-Gaussian constraint with the self-predicted priors, EVA significantly alleviates the latent distribution gap between prior and posterior which is typically observed in conventional VAEs, and leads to high-fidelity ancestral sampling for sequential data generation. Extensive experiments on image and sound synthesis demonstrate that EVA achieves competitive generation quality with autoregressive diffusion baselines despite its much faster inference time.