Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces TALON, a test-time adaptation framework for on-the-fly category discovery (OCD) that addresses limitations of fixed feature extractors and feature quantization in existing methods. TALON incorporates semantic-aware prototype updates and stable test-time encoder updates to dynamically refine class prototypes and integrate new information into the model's parameter space during the test phase. By also introducing margin-aware logit calibration during training, TALON enlarges inter-class margins and improves intra-class compactness, reserving embedding space for future class discovery. Experiments on OCD benchmarks demonstrate that TALON significantly outperforms state-of-the-art hash-based approaches, improving novel-class accuracy and mitigating category explosion.
Forget freezing your feature extractor: TALON unlocks on-the-fly category discovery by continuously learning from unlabeled data during test time, outperforming fixed-knowledge methods.
On-the-fly category discovery (OCD) aims to recognize known categories while simultaneously discovering novel ones from an unlabeled online stream, using a model trained only on labeled data. Existing approaches freeze the feature extractor trained offline and employ a hash-based framework that quantizes features into binary codes as class prototypes. However, discovering novel categories with a fixed knowledge base is counterintuitive, as the learning potential of incoming data is entirely neglected. In addition, feature quantization introduces information loss, diminishes representational expressiveness, and amplifies intra-class variance. It often results in category explosion, where a single class is fragmented into multiple pseudo-classes. To overcome these limitations, we propose a test-time adaptation framework that enables learning through discovery. It incorporates two complementary strategies: a semantic-aware prototype update and a stable test-time encoder update. The former dynamically refines class prototypes to enhance classification, whereas the latter integrates new information directly into the parameter space. Together, these components allow the model to continuously expand its knowledge base with newly encountered samples. Furthermore, we introduce a margin-aware logit calibration in the offline stage to enlarge inter-class margins and improve intra-class compactness, thereby reserving embedding space for future class discovery. Experiments on standard OCD benchmarks demonstrate that our method substantially outperforms existing hash-based state-of-the-art approaches, yielding notable improvements in novel-class accuracy and effectively mitigating category explosion. The code is publicly available at \textcolor{blue}{https://github.com/ynanwu/TALON}.