Search papers, labs, and topics across Lattice.
This paper introduces a Bayesian Region of Practical Equivalence (ROPE)-based framework with binning-based statistics to evaluate the practical equivalence of synthetic pre-crash scenarios to real-world data for driving automation system safety assessment. The framework uses two novel binning-based statistics to quantify meaningful distributional differences between datasets, addressing the limitations of conventional significance testing. A case study validating synthetic rear-end pre-crash datasets demonstrates the framework's ability to provide quantitative assessments and diagnostic insights into dataset divergence.
Stop relying on significance tests that only find differences: this Bayesian framework tells you if your synthetic data is *practically equivalent* to real-world data for your specific safety assessment task.
The representativeness of synthetic pre-crash scenarios is crucial for assessing the safety impact of Driving Automation Systems through virtual simulations. However, a gap remains in the robust evaluation of synthetic pre-crash scenarios'practical equivalence to their real-world counterparts; that is, whether they are similar enough for the intended assessment purpose. Conventional significance testing is inadequate, as it focuses on detecting differences rather than establishing practical equivalence. This study addresses the research gap by extending our previous work on a Bayesian Region of Practical Equivalence (ROPE)-based equivalence testing framework by introducing a binning-based approach to define appropriate statistics and equivalence criteria. Two binning-based statistics are proposed to measure practically meaningful distributional differences between datasets in the context of safety impact assessment. The framework's applicability is demonstrated through a case study, which tests the practical equivalence of two synthetic rear-end pre-crash datasets with a previously developed reference dataset in the context of the safety impact assessment of an Automatic Emergency Braking system. The results show that the framework provides informative quantitative assessments of practical equivalence as well as diagnostic insights into the divergence of datasets. Although the demonstration focuses on rear-end pre-crash scenarios, the framework is generic and extensible to broader validation contexts, providing an interpretable and principled basis for practical equivalence assessment across diverse synthetic data applications.
