Track synthesis, materials and catalysis as it is published.
Search papers, labs, and topics across Lattice.
To bridge the gap between rigid physics-based priors and purely implicit attention, the authors introduce Minkowski Positional Encoding (MinkowskiPE) for unified spatiotemporal representation learning. The method parameterizes Lorentz transformations on query and key tokens via joint spacetime coordinates, making standard dot-product attention strictly translation-invariant and dependent solely on relative spacetime intervals. Across benchmarks, MinkowskiPE sweeps all nine multi-trajectory molecular dynamics evaluations and achieves a 9.9% MSE reduction on KTH video prediction while using nearly an order of magnitude fewer parameters than prior baselines.
Embedding Lorentz transformations directly into query-key attention cuts video prediction error by nearly 10% using a tenth of the parameters, proving that explicit relativistic spacetime geometry can dramatically outperform purely learned positional encodings across physical scales.
Modeling spatiotemporal coupling is a key challenge in building physical intelligence across scales, from microscopic to macroscopic. Existing models capture such structure broadly through physics-motivated dynamical formulations or learning-motivated architectures. The former provide stronger priors but may constrain flexibility, whereas the latter are more flexible but leave the spatiotemporal coupling largely implicit. We therefore seek an approach that combines flexible learning with an explicit geometric bias for jointly modeling time and space. To this end, we propose Minkowski Positional Encoding (MinkowskiPE), which uses joint temporal and spatial coordinates to parameterize Lorentz transformations applied to query and key features. With MinkowskiPE, the query-key attention score depends on position only through the relative spacetime displacement between the two tokens and is therefore invariant to global translation of the coordinates. This paradigm retains the standard dot-product attention interface and remains compatible with efficient attention implementations. We evaluate MinkowskiPE on microscopic molecular dynamics and macroscopic video prediction tasks, achieving the best results on all nine multi-trajectory molecular evaluations and reducing KTH video-prediction MSE by 9.9% relative to the best baseline while using roughly one-tenth as many parameters.