Search papers, labs, and topics across Lattice.
This paper introduces GKDT, a novel General Keypoint Detection Transformer built on a large-scale unified dataset called MegaKPT, which contains over 1.3 million annotated object instances. The model leverages advanced training strategies, including mix-modal prompted training and dynamic importance sampling, to achieve over 90% PCK@0.1 accuracy across diverse categories in both seen and unseen object tests. This work addresses the critical need for a robust keypoint detection framework that can adapt to open-domain scenarios, significantly enhancing the capabilities of computer vision systems.
Achieving over 90% accuracy in keypoint detection across diverse categories, GKDT sets a new standard for open-domain recognition tasks.
With the emergence of various pre-trained vision and language models, computer vision is shifting from narrow-domain to open-domain recognition. The construction of a more powerful yet general keypoint detection (GKD) model to support diverse tasks has become increasingly important in the field. To this end, we firstly present a large-scale unified keypoint dataset called MegaKPT. The dataset is composed of over 1.3 million diverse object instances from twenty-nine existing datasets, and enjoys high-quality unified annotations with keypoint text descriptions. Based on MegaKPT, we develop GKDT, a simple, flexible and powerful DINOv3 based Transformer model for General Keypoint Detection. Our GKDT supports visual prompts, text prompts, or both. To enhance model training, we also propose a suite of useful strategies such as mix-modal prompted training and dynamic importance sampling. By testing over 22 test sets with seen or unseen objects, our single GKDT model shows strong performance and generality in detecting keypoints on broad categories, with most categories over 90\% PCK@0.1 accuracy, offering high practical applicability to real-world problems. The dataset, models, and codes will be released at https://github.com/AlanLuSun/General-Keypoint-Detection.
