Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates whether out-of-distribution (OOD) degradation can be predicted solely from source-side training dynamics, without observing OOD data. Using a shortcut-learning setup, a logistic regression model successfully predicts OOD failure by analyzing temporal summaries of source-side quantities like confidence and entropy, outperforming predictions based on current values or training time alone. The findings demonstrate that these source-side dynamics retain predictive power even when transferred across different model architectures (CNN to MLP), offering a proof of principle for early OOD failure detection.
Source-side training dynamics, not just current values or training time, prospectively predict OOD degradation, even across model architectures.
We study whether persistent out-of-distribution (OOD) degradation can be predicted before it is directly observed using only source-side training dynamics. In a controlled shortcut-learning setting, a simple logistic regression predictor develops a clear prospective signal, while training time alone does not. Temporal summaries of the source-side quantities are substantially more informative than their current values. When transferred without additional training from a CNN to an MLP, confidence and entropy dynamics retain substantial predictive information. These results provide a proof of principle that source-side training dynamics can contain an early warning signal for future OOD failure.