Search papers, labs, and topics across Lattice.
This paper introduces a transformer-based autoregressive sampler for 2D spin systems that generates groups of spins per step, improving sampling efficiency. They further enhance efficiency by modeling approximated probabilities. Despite higher computational cost per step compared to dense or CNN-based methods, the approach enables sampling larger Ising models (up to 180x180) and achieves a 20x improvement in Effective Sample Size compared to prior neural samplers on 128x128 Ising models at critical temperature.
Transformers, typically considered inefficient for spin system sampling, can now outperform CNN-based samplers by generating groups of spins, unlocking larger system sizes and higher effective sample sizes.
Autoregressive Neural Networks based on dense or convolutional layers have recently been shown to be a viable strategy for generating classical spin systems. Unlike these methods, sampling with transformers is commonly considered to be computationally inefficient. In this work, we propose a novel approach to transformer-based neural samplers in which we generate not a single spin per step but groups of spins. As an additional improvement, we construct a model of approximated probabilities, further improving the efficiency of the algorithm. Despite our approach being computationally heavier than dense networks or CNN-based approaches, we were able to sample larger systems of up to $180 \times 180$ spins in case of the Ising model. The Effective Sample Size of our sampler is $\sim 20$ times larger than that of the previous state-of-the-art neural sampler when trained for the $128 \times 128$ Ising model at critical temperature. Finally, we also test our algorithm on the 2D Edwards-Anderson model, where we train $64\times 64$ spin systems.
