Search papers, labs, and topics across Lattice.
The paper introduces SLALOM, a framework for validating LLM-based social simulations by assessing the fidelity of the simulation's trajectory, rather than just the final outcome. SLALOM uses Dynamic Time Warping (DTW) to align simulated multivariate time series with empirical data, evaluating whether the simulation passes through predefined "SLALOM gates" representing key phases of the social phenomenon. This approach provides a quantitative metric for structural realism, enabling researchers to distinguish plausible social dynamics from noise.
LLM-based social simulations can now be rigorously validated throughout their entire lifecycle, not just at the end, thanks to a new framework that aligns simulated trajectories with real-world data.
Large Language Model (LLM) agents offer a potentially-transformative path forward for generative social science but face a critical crisis of validity. Current simulation evaluation methodologies suffer from the "stopped clock" problem: they confirm that a simulation reached the correct final outcome while ignoring whether the trajectory leading to it was sociologically plausible. Because the internal reasoning of LLMs is opaque, verifying the "black box" of social mechanisms remains a persistent challenge. In this paper, we introduce SLALOM (Simulation Lifecycle Analysis via Longitudinal Observation Metrics), a framework that shifts validation from outcome verification to process fidelity. Drawing on Pattern-Oriented Modeling (POM), SLALOM treats social phenomena as multivariate time series that must traverse specific SLALOM gates, or intermediate waypoint constraints representing distinct phases. By utilizing Dynamic Time Warping (DTW) to align simulated trajectories with empirical ground truth, SLALOM offers a quantitative metric to assess structural realism, helping to differentiate plausible social dynamics from stochastic noise and contributing to more robust policy simulation standards.
