Search papers, labs, and topics across Lattice.
This paper investigates the application of Relative Positional Encoding (RPE) as an additive bias within Transformer architectures to address the Team Orienteering Problem. By incorporating pairwise spatial relationships into the attention mechanism, the authors enable the transformer encoder to generate more effective spatial-aware graph embeddings, leading to improved route estimations by the decoder. Experimental results reveal that this approach consistently outperforms traditional Transformer architectures, achieving better rewards and reduced optimality gaps across instances of up to 100 nodes.
Encoding spatial relationships in Transformers can significantly boost performance in complex routing problems, achieving better solutions than conventional methods.
This paper explores Relative Positional Encoding (RPE) as an additive bias in Transformer architectures to solve the Team Orienteering Problem. By embedding in the attention mechanism pairwise spatial relationships among nodes of the graph that represents the routing problem, the transformer encoder can compute a richer spatial-aware graph embedding that allows the decoder to estimate better routes. Experimental results involving instances up to 100 nodes demonstrate consistent improvements in collected rewards and optimality gaps over vanilla Transformer architectures used by other state-of-the-art works. These findings highlight that explicit relational modeling significantly enhances scalability and generalization for complex combinatorial optimization.
