Search papers, labs, and topics across Lattice.
The paper introduces the Physics Transformer, a novel Transformer architecture tailored for predicting partial differential equations (PDEs) by treating physical fields as continuous functions and employing a function-projection approach. This method partitions discretized fields into locality-preserving spatial patches, allowing for the dynamic learning of adaptive local basis functions that generate compact physics tokens. Extensive experiments reveal that the Physics Transformer not only captures fine-grained physical structures but also outperforms existing methods in predictive accuracy across various PDE benchmarks, including complex three-dimensional simulations.
Capturing fine-scale physical structures in PDE predictions is now achievable with a novel function-projection approach that outperforms traditional methods.
Transformer architectures have attracted increasing attention for solving partial differential equations (PDEs), owing to their flexibility in handling irregular discretizations and their ability to capture long-range physical dependencies. However, unlike discrete language tokens or fixed-resolution image patches, observed physical fields are finite samples of underlying infinite-dimensional functions. Consequently, effectively applying Transformers to PDEs requires a tokenizer that respects the functional nature of physical fields and constructs physically expressive tokens from arbitrary discretizations.To this end, we propose \methodname{Physics Transformer}, a function-projection-based Transformer architecture for physical field prediction. Physics Transformer treats a physical field as a continuous function and partitions its discretization into locality-preserving spatial patches. Within each patch, it dynamically learns a set of adaptive local basis functions and projects the sampled field onto these bases to obtain compact physics tokens. The resulting tokens capture diverse latent physical states while preserving fine-scale spatial structures, enabling efficient global interaction through factorized attention across space and physical states. The projected representation further supports efficient decoding at arbitrary query locations. Extensive experiments on diverse benchmarks, ranging from two-dimensional PDE dynamics to industrial-scale three-dimensional CFD simulations, demonstrate that Physics Transformer accurately captures fine-grained physical structures and achieves state-of-the-art predictive performance. These results establish function projection as a practical and effective foundation for designing Transformer architectures for PDE solving.
