Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
GPUPhysBench establishes an execution-based evaluation suite of 50 GPU physics simulation tasks spanning fluids, deformable bodies, and granular materials to test coding agents under fixed time budgets. Evaluating numerical simulation workloads exposes critical model deficiencies in managing irregular memory access patterns, thread synchronization, and iterative numerical solvers. Although frontier coding agents achieve 100% functional pass rates, the top-performing agent matches expert-optimized runtime ($\ge 0.9\times$ reference speed) on only 22% of tasks, suffering massive performance regressions on collision detection and constraint solving.
Frontier coding agents can now write 100% functionally correct GPU physics simulations, yet they match human-expert performance on just 22% of tasks—revealing a stark capability gap between generating working code and producing performant systems.
Writing fast GPU code for physical simulation is difficult: implementations must preserve numerical accuracy while handling irregular data access, synchronization, and iterative solvers. We introduce GPUPhysBench, a benchmark of 50 tasks testing whether coding agents can meet these demands. Tasks cover fluids, deformable solids, and granular materials, from individual simulation operators to complete simulators. Agents write, compile, test, and optimize GPU code with access to a NVIDIA GPU under fixed time budgets. We report pass rates and runtime performance relative to expert-optimized reference implementations. In a single-attempt evaluation of six frontier model-harness pairs, the two strongest pass all 50 tasks, but even the fastest reaches at least 0.9 the reference speed on only 22% of them, and no submission is more than 5% faster than the reference. The largest gaps arise in collision detection, constraint solving, and iterative solvers. GPUPhysBench brings physical simulation workloads to coding-agent evaluation, testing both the ability to implement numerical methods correctly and the ability to make them run efficiently.