Search papers, labs, and topics across Lattice.
This study investigates the behavior of $β$-VAEs, particularly how increasing regularization strength influences the collapse of low-utility latent coordinates in nonlinear settings. The authors demonstrate that while the collapse order in linear Gaussian VAEs aligns perfectly with reconstruction utilities, this alignment shifts in fully connected nonlinear VAEs, indicating a more complex relationship between latent dimensions and utility. Despite these shifts, the fundamental ordering of latent dimensions remains intact, revealing a tradeoff between utility concentration and tail fidelity as model depth increases.
Nonlinear $β$-VAEs reveal a surprising tradeoff where deeper models concentrate utility but compromise the fidelity of less significant dimensions.
In a $β$-VAE, increasing the regularization strength acts as a spectral cutoff by collapsing low-utility latent coordinates. In the linear Gaussian VAE, the collapse order matches the ranking of reconstruction utilities exactly, because both are set by the PCA spectrum. We ask which parts of this picture survive in fully connected nonlinear VAEs trained on WorldClim. We find that nonlinear interactions shift and broaden collapse onsets, so thresholds no longer coincide exactly with utilities. However, the common ordering is preserved over the resolved ranks, so the spectral cutoff still acts as a utility cutoff and the effective-description logic carries through. The resulting effective-dimension curves reveal a head--tail tradeoff: increasing depth concentrates utility into the first few coordinates but worsens tail fidelity.
