Search papers, labs, and topics across Lattice.
This paper introduces a novel training algorithm called Parallel Trajectory Tempering (PTT) for Energy-Based Models (EBMs), addressing the challenge of poor Markov Chain Monte Carlo mixing that often hampers their reliability. By leveraging the continuity of the optimization path, PTT facilitates stable and efficient training on multimodal and data-scarce scientific datasets, achieving results comparable to Persistent Contrastive Divergence while providing additional benefits like direct thermalization time estimates. Experiments demonstrate that PTT not only outperforms existing EBM training methods but also surpasses state-of-the-art deep generative models in generating higher-quality samples and exhibiting greater robustness against overfitting.
PTT transforms EBM training by enabling practical equilibrium maximum-likelihood training that consistently yields superior sample quality and robustness.
Energy-Based Models (EBMs) provide an interpretable framework for generative modeling of scientific data, but poor Markov Chain Monte Carlo mixing often limits their reliability. We introduce a training algorithm based on Parallel Trajectory Tempering (PTT), which exploits the continuity of the optimization path to maintain equilibrium sampling throughout learning. This enables stable and fast training on highly multimodal and data-scarce scientific datasets. Combined with reservoir sampling and adaptive optimization, PTT has a computational cost comparable to Persistent Contrastive Divergence, making it a practical replacement for standard training methods. It also provides direct estimates of thermalization times, equilibrium samples from trained models, and accurate log-likelihoods at essentially no additional cost. Experiments on Restricted Boltzmann Machines show that PTT consistently outperforms existing EBM training approaches. On discrete tabular data, it also surpasses state-of-the-art deep generative models, yielding higher-quality samples and greater robustness to overfitting and limited data. Our results make equilibrium maximum-likelihood training of EBMs practical and computationally efficient.
