Search papers, labs, and topics across Lattice.
This paper introduces ControlledShifts, a novel framework and benchmark suite designed to standardize the evaluation of robustness in trajectory prediction models under distribution shifts. By systematically re-splitting existing datasets into in-distribution and out-of-distribution partitions, the authors create a comprehensive method for assessing model performance across key topological and behavioral shifts. The results reveal significant disparities in how transformer-based architectures manage prediction quality and stability, highlighting the need for standardized evaluation in this critical area of autonomous driving safety.
Transformer-based models exhibit stark differences in handling distribution shifts, revealing vulnerabilities that could jeopardize autonomous driving safety.
Trajectory prediction is central to safety in autonomous driving, yet learning-based predictors tend to degrade sharply when encountering scenarios poorly represented by their training data. Many methods attempt to mitigate distribution shift degradation through data-centric or test-time adaptation approaches; however, they are typically validated along fragmented axes of generalization, leaving the field without a standardized way to compare robustness across shifts a model may encounter. To address this, we introduce ControlledShifts, a framework and benchmark suite that systematically re-splits existing trajectory datasets into in-distribution (seen) and out-of-distribution (unseen) partitions, via a shared characterization-and-splitting formulation, in which a characterization function fixes the axis of variation a benchmark probes and a splitting function fixes how the tail of that axis is withheld. The suite comprises three benchmarks targeting key topological and behavioral distribution shifts. Furthermore, to aggregate multi-dimensional performance metrics across these benchmarks, we propose a unified robustness score that evaluates models along two complementary dimensions: prediction quality (relative performance gain) and prediction stability (performance preservation under shift). We showcase ControlledShifts by benchmarking prominent transformer-based architectures, exposing critical differences in how models of varying capacities handle latent relevance and environmental structure.
