Search papers, labs, and topics across Lattice.
This paper investigates geometric regularization techniques for autoencoder-based reduced-order models with neural ODE latent dynamics applied to the advection-diffusion-reaction equation. The study evaluates four regularization approaches during autoencoder pre-training: near-isometry regularization, stochastic decoder gain penalty, directional curvature penalty, and Stiefel projection of the first decoder layer. The key finding is that Stiefel projection consistently improves latent dynamics conditioning and long-horizon rollout performance, while other regularization methods often hinder subsequent latent-dynamics training.
Latent-space geometry matters more than decoder smoothness when training autoencoder reduced-order models with neural ODE dynamics, challenging the intuition that smoother decoders always lead to better performance.
We investigate geometric regularization strategies for learned latent representations in encoder--decoder reduced-order models. In a fixed experimental setting for the advection--diffusion--reaction (ADR) equation, we model latent dynamics using a neural ODE and evaluate four regularization approaches applied during autoencoder pre-training: (a) near-isometry regularization of the decoder Jacobian, (b) a stochastic decoder gain penalty based on random directional gains, (c) a second-order directional curvature penalty, and (d) Stiefel projection of the first decoder layer. Across multiple seeds, we find that (a)--(c) often produce latent representations that make subsequent latent-dynamics training with a frozen autoencoder more difficult, especially for long-horizon rollouts, even when they improve local decoder smoothness or related sensitivity proxies. In contrast, (d) consistently improves conditioning-related diagnostics of the learned latent dynamics and tends to yield better rollout performance. We discuss the hypothesis that, in this setting, the downstream impact of latent-geometry mismatch outweighs the benefits of improved decoder smoothness.
