Search papers, labs, and topics across Lattice.
This paper introduces STORM, an open-source library that employs Remote Direct Memory Access (RDMA) to enhance the scalability of Monte Carlo particle transport simulations in astrophysical contexts. By replacing traditional two-sided MPI communication with a lock-free, one-sided operation, STORM achieves significant reductions in per-iteration overhead, allowing for efficient scaling on unstructured meshes. The results demonstrate that STORM can sustain over 97% weak-scaling and 88% strong-scaling efficiency across thousands of cores, outperforming conventional MPI approaches by up to 1.41 times in specific benchmarks.
RDMA-based communication in STORM achieves unprecedented scaling efficiency for Monte Carlo simulations, reducing overhead and enabling high-fidelity astrophysical modeling at scale.
Monte Carlo particle transport enables high-fidelity astrophysical radiation and neutrino simulations - from core-collapse supernovae and neutron-star mergers to accretion flows - by handling multidimensional geometries, frequency dependence, and moving media without angular discretization. However, inter-rank communication limits scalability on unstructured meshes: standard two-sided MPI requires receivers to post receives and poll completions, creating per-iteration progress overhead that grows with the number of communication partners. Such problems have not demonstrated high scaling efficiency at $O(10^4)$ cores. We present STORM (Scalable Transport via One-sided Remote Memory), an open-source library for Monte Carlo transport on general meshes, physics, and boundary conditions. STORM provides a lock-free, mesh-independent communication layer that replaces MPI's matched-send/receive semantics with Remote Direct Memory Access (RDMA) - one-sided operations that write directly into a remote rank's memory without involving its CPU. Each rank pair shares a single-producer, single-consumer ring buffer; RDMA writes transfer particles while receivers remain passive. A two-sided MPI backend provides a portable fallback. In an adversarial uniform-emission benchmark, the RDMA backend sustains $>97\%$ weak-scaling and $>88\%$ strong-scaling efficiency up to 13,440~cores (112~cores per network adapter), with $1.14$-$1.27\times$ speedups over the two-sided alternative. In a Hohlraum IMC benchmark at 4480 ranks, it is $1.41\times$ faster because MPI progress overhead is reduced by $6.1\times$. By decoupling communication from physics models and mesh representations, STORM removes a barrier to scaling Monte Carlo transport in astrophysical multiphysics codes, enabling coupled radiation-hydrodynamics with energy- and angle-resolved photon or neutrino transport on dynamically evolving meshes at scale.
