Search papers, labs, and topics across Lattice.
This paper introduces PuRF, a novel method for multi-label test-time adaptation (MLTTA) of vision-language models that addresses the challenges of dominant-label bias and unreliable regional cues under distribution shifts. By employing a two-step purification process鈥攔egion purification to identify reliable regions and cache purification to enhance cache representation鈥擯uRF enables fine-grained alignment for improved multi-label recognition. Experimental results show that PuRF achieves a significant 4.05% mean Average Precision (mAP) improvement on ViT-B/32 across five datasets, surpassing state-of-the-art methods in the domain.
Region purification in MLTTA reveals reliable cues that dramatically enhance multi-label recognition performance, achieving a 4.05% mAP boost over existing methods.
Test-time adaptation (TTA) has been widely explored in single-label recognition, effectively mitigating distribution shifts, especially when combined with vision-language models. However, real-world images often contain multiple objects, while the more practical multi-label test-time adaptation (MLTTA) has received little attention so far. Recent cache-based TTA methods have shown promising efficiency and effectiveness, yet directly extending them to multi-label scenarios suffers from a one-to-many mapping problem: a shared global representation entangling co-occurring objects is stored as class-wise cache prototypes, inducing dominant-label bias and compromised cache calibration. While introducing region-level cues helps isolate class-specific evidence, such regional evidence can also be unreliable under distribution shifts, making its identification and utilization non-trivial. To address these issues, we introduce PuRF, a novel PuRiFication-driven cache-based method for multi-label test-time adaptation of vision-language models. Specifically, PuRF first performs region purification to identify reliable regions, providing comprehensive regional cues for multi-label recognition and enabling fine-grained alignment. Based on these purified regions, PuRF conducts cache purification to enhance cache representation and adaptability, where episodic purification builds a discriminative region-based cache, and temporal refreshing further promotes long-term cache adaptability. Experiments demonstrate that PuRF consistently outperforms state-of-the-art methods, achieving a notable 4.05% mAP improvement on ViT-B/32 across five datasets.
