Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces higher-order positional encodings that enrich graph representations by directly incorporating topological information from simplicial complexes, overcoming the limitations of existing methods that rely solely on pairwise graph structure. Theoretical analysis proves that these encodings enable node-level operators to mix graph Laplacian frequencies beyond the capabilities of standard spectral filters. Experimental results on Graph Transformers show improved predictive performance by leveraging this lifted incidence structure without modifying the underlying graph learning architecture.
Standard GNNs and Transformers can now capture complex multi-way interactions by simply adding higher-order positional encodings derived from clique complexes, boosting performance on tasks requiring rich relational understanding.
Many real-world systems exhibit higher-order interactions among groups of entities that cannot be captured by pairwise relationships alone. Graph Transformers and Graph Neural Networks increasingly rely on positional encodings to enrich graph representations, yet existing positional encodings are computed solely from the original graph and therefore cannot directly capture observed higher-order interactions. Topological Deep Learning addresses this limitation by lifting graphs to simplicial complexes, but typically requires performing message passing or attention on higher-order neural network representations. We introduce a representation learning paradigm that enriches graph representations with higher-order topology through positional encodings, enabling standard graph learning models to exploit lifted incidence structure without modifying the backbone. We derive a theoretical characterization of the expressivity of higher-order positional encodings, proving that node-level operators induced by higher-order lifts can mix graph Laplacian frequencies in ways that scalar graph spectral filters cannot. Guided by this theory, we instantiate higher-order positional encodings using Hodge Laplacians derived from clique complexes. Experiments with Graph Transformers on ZINC and controlled synthetic benchmarks demonstrate improvements in predictive performance, while a fixed-1-skeleton experiment shows that the pipeline can transmit higher-order information when cells are supplied independently of the graph. Together, our results establish higher-order positional encodings as a principled bridge between graph positional encodings and topological deep learning.