Search papers, labs, and topics across Lattice.
To overcome error accumulation and catastrophic forgetting during test-time adaptation (TTA), the authors introduce distilled image prototypes (DIP) as compact, synthetic representations of source knowledge. By dynamically re-encoding these synthetic images through the adapting model, the framework maintains synchronized prototype features for replay while calibrating target-domain uncertainty estimates against stable source representations. This dual-purpose mechanism consistently outperforms state-of-the-art TTA baselines across standard benchmarks, exhibiting particular robustness against severe domain shifts.
Re-encoding a compact set of distilled synthetic images during test-time adaptation dynamically realigns source anchors with shifting model weights, eliminating both catastrophic forgetting and pseudo-label drift under severe distribution shifts.
Test-Time Adaptation (TTA) enhances the robustness of models against distribution shifts but faces two critical challenges: error accumulation from noisy pseudo-labels and catastrophic forgetting of source knowledge. Uncertainty-based approaches designed to mitigate error accumulation often yield overconfident or computationally expensive estimates, while strategies intended to prevent forgetting via prototype replay rely on static representations that easily become misaligned as the model adapts. To address these issues, this paper proposes a novel framework, Distilling Image Prototype for Guided Test-Time Adaptation (DIPTTA). The core of the proposed approach is the introduction of a Distill Image Prototype (DIP), a compact set of synthetic images that serves as a dynamic and regenerative anchor of source knowledge. This prototype enables a dynamic feature replay mechanism that continuously generates feature prototypes aligned with the current state of the model, thus effectively preventing catastrophic forgetting. Furthermore, the DIP anchors a source-calibrated uncertainty estimation method, which provides a less biased measure of sample reliability by leveraging stable source knowledge, thereby robustly suppressing error accumulation. Extensive experiments on multiple benchmarks demonstrate that DIPTTA significantly outperforms state-of-the-art methods, particularly under severe domain shifts. The source code is available at https://github.com/LiwenWang919/DIPTTA.
