Search papers, labs, and topics across Lattice.
The paper investigates how neural forecasters represent the latent geometry of dynamical systems by introducing anchor-based, geometry-agnostic relative embeddings to remove rotational and scaling ambiguities. They find reproducible family-level structure in latent spaces, with MLPs aligning with MLPs, RNNs with RNNs, and transformers/echo-state networks exhibiting weaker alignment despite strong forecasting. The study reveals a general correlation between alignment and forecasting accuracy, while also demonstrating instances of high accuracy with low alignment, suggesting alignment is not strictly necessary for accurate forecasting.
Even though transformers and echo-state networks can accurately forecast dynamical systems, their internal representations are surprisingly misaligned compared to MLPs and RNNs.
Neural networks can accurately forecast complex dynamical systems, yet how they internally represent underlying latent geometry remains poorly understood. We study neural forecasters through the lens of representational alignment, introducing anchor-based, geometry-agnostic relative embeddings that remove rotational and scaling ambiguities in latent spaces. Applying this framework across seven canonical dynamical systems - ranging from periodic to chaotic - we reveal reproducible family-level structure: multilayer perceptrons align with other MLPs, recurrent networks with RNNs, while transformers and echo-state networks achieve strong forecasts despite weaker alignment. Alignment generally correlates with forecasting accuracy, yet high accuracy can coexist with low alignment. Relative geometry thus provides a simple, reproducible foundation for comparing how model families internalize and represent dynamical structure.
