Search papers, labs, and topics across Lattice.
This study investigates the interplay between model compression and test-time adaptation (TTA) in deep neural networks, focusing on how structured compression impacts a model's adaptability to distribution shifts. By employing ResNet-18 and ViT-Base on CIFAR-10-C and ImageNet-C, the authors reveal that while compressed models maintain high accuracy during supervised adaptation, their TTA performance deteriorates with increased compression due to diminished representational diversity. The findings emphasize the importance of developing compression techniques that maintain adaptability, as the choice of compression method significantly influences TTA outcomes.
Compressed models may achieve high accuracy but suffer a dramatic drop in adaptability during test-time, revealing a critical trade-off in model deployment.
Deep neural networks deployed in the wild must be both efficient and adaptable, requiring model compression and test-time adaptation (TTA). While both are well studied in isolation, their interaction remains poorly understood. We systematically analyze how structured compression affects a model's ability to adapt under distribution shift. Using ResNet-18 and ViT-Base on CIFAR-10-C and ImageNet-C, we evaluate multiple compression methods combined with standard TTA techniques. We introduce a diagnostic framework that examines representational expressivity and adaptation subspace compatibility. Our results reveal a consistent gap: although compressed models retain high accuracy under supervised adaptation, their TTA performance degrades significantly with increasing compression. We show that this stems from reduced representational diversity and structural constraints that limit recoverability. These effects strongly depend on the compression method, highlighting the need to design compression strategies that preserve adaptability.
