Search papers, labs, and topics across Lattice.
This paper demonstrates that posterior collapse in β-VAEs acts as a form of automatic spectral pruning, where latent modes are selectively disabled based on their contribution to reconstruction relative to the β parameter. The authors use Landau stability analysis to show that latent modes decouple from least to most useful as β increases, resulting in a cascade of collapses. They introduce a latent-rescaling-invariant order parameter to rank active latent modes and validate their theoretical predictions on the WorldClim dataset, showing agreement with mean-field theory in the linear Gaussian case.
Posterior collapse isn't a bug in VAEs, it's a feature that automatically prunes latent modes by importance, revealing a utility spectrum akin to PCA.
We show that posterior collapse in $β$-VAEs implements automatic spectral pruning. A latent mode collapses if its contribution to reconstruction is below the cutoff set by $β$. Equilibrium solutions with different $β$ thus reveal a cascade of collapses as latent modes decouple from least to most useful. We derive this as a consequence of the loss via a Landau stability analysis. We define a latent-rescaling-invariant order parameter that ranks active latent modes and whose collapse thresholds identify which effective variables to inspect first. In the linear Gaussian case, the collapse spectrum, utility spectrum, and normalized PCA spectrum coincide, and each collapse follows a mean-field law. We test these predictions on the WorldClim dataset.
