Track the frontier, from condensed matter to cosmology.
Search papers, labs, and topics across Lattice.
This paper addresses the under-explored area of tokenization for multi-modal foundation models in High Energy Physics (HEP) by framing it as a rate-distortion problem to learn compressed, discrete representations of detector data. The approach is validated across four modalities from two simulated detectors, yielding four design rules for tokenization that generalize to set-based data. Experiments show that this precision tokenization preserves crucial information for tasks like jet $p_T$ regression, particle flow reconstruction, and flavor tagging, achieving compression factors of 4-11$\times$ with minimal impact on jet $p_T$ resolution (<1%).
HEP foundation models can achieve 4-11x data compression without sacrificing critical physics information, paving the way for more efficient multi-modal analysis.
Foundation models have recently gained significant attention in fields such as high energy physics (HEP). The tokenizers that convert detector data into discrete inputs for such models, however, are comparatively unexplored. This paper addresses this gap by learning discrete and compressed tokenized representations of HEP datasets. We demonstrate the approach on four modalities covering two different simulated detectors that vary in complexity. Treating tokenization as a rate-distortion problem, a systematic hyperparameter scan yields four design rules for tokenizing HEP data that hold across all set-based modalities studied. The cost of tokenization is evaluated on tasks including jet $p_T$ regression, particle flow reconstruction, and flavor tagging. We find that tokenization preserves much of the information needed for these tasks at compression factors of approximately 4-11$\times$ while keeping the jet $p_T$ resolution below 1%. We also present heptokens, an open-source library for tokenizing HEP data.
Weizmann