Search papers, labs, and topics across Lattice.
The paper introduces Iterative Self-Training with Class-Aware Text-to-Image Synthesis (IST-CATS), a framework that leverages class-aware text-to-image synthesis (CATS) and iterative self-training (IST) to generate high-quality synthetic data for training visual models. CATS uses a class-aware chain approach to create detailed text prompts for a diffusion model, generating diverse images with distinguishable objects, which are then pseudo-labeled and refined using a feature similarity-based filtering mechanism (LabFilt). IST-CATS demonstrates improved performance in object detection and semantic segmentation by iteratively enhancing vision models and refining pseudo labels with class-adaptive techniques at the pixel and object levels.
Synthetic data quality takes a leap forward: IST-CATS generates images so good, they outperform traditional semi/weakly-supervised methods in object detection and semantic segmentation.
Generative models are widely used to produce synthetic images with annotations, alleviating the burden of image collection and annotation for training deep visual models. However, challenges such as limited image diversity, noisy pseudo labels, and domain gaps between synthetic and real images often undermine their effectiveness in downstream visual tasks. This paper introduces the Iterative Self-Training with Class-Aware Text-to-Image Synthesis (IST-CATS) framework, which addresses these challenges by integrating a class-aware text-to-image synthesis (CATS) component with an iterative self-training (IST) strategy. CATS innovatively introduces a class-aware chain approach to generate detailed descriptions. These descriptions act as prompts for a diffusion model, enabling the creation of a diverse of images accompanied by distinguishable objects against the background. The generated images can be easily pseudo-labeled by an unsupervised instance segmentation method, and then noisy pseudo labels can be effectively purified by a novel feature similarity-based filtering mechanism. The generated images underpin our IST, which progressively enhances vision models and refines pseudo labels through self-training and our proposed label filtering strategy (LabFilt). LabFilt meticulously improves the quality of pseudo labels by employing class-adaptive techniques at both the pixel and object levels, ensuring refined pseudo-label accuracy. IST-CATS demonstrates superior performance in object detection and semantic segmentation compared to traditional synthetic and semi/weakly-supervised methods, effectively addressing data collection and annotation challenges.
