Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces QueenVIS, a novel query-centric framework that enhances image-only training for video instance segmentation (VIS) by enriching Mask2Former queries with auxiliary heads for feature and center prediction. By addressing the identified bottleneck of object query quality, QueenVIS maintains temporal identity through a training-free query-propagation and memory-bank scheme, achieving significant performance improvements on benchmark datasets. The results show that QueenVIS outperforms the previous image-only method, MinVIS, by up to 6.7 AP on YouTube-VIS, demonstrating that enhancing query stability is crucial for effective VIS without relying on video-level supervision.
Strengthening object query stability in image-only training can yield performance on par with video-supervised methods, achieving up to +10.3 AP improvements without processing any video clips during training.
Video instance segmentation (VIS) requires models to detect, segment, and track object identities across frames, and most methods enforce temporal consistency through video-level supervision. Image-only training approaches, with MinVIS as one prominent example, have challenged this assumption, reaching competitive VIS without video training by treating frames as independent images and associating instances only at inference. The field has nonetheless moved toward ever more elaborate video-trained trackers, which depend on costly identity-consistent annotations, leaving the image-only direction under-explored. A diagnostic analysis identifies object query quality as the bottleneck: queries trained only to localize objects within a frame drift apart across frames and destabilize association. QueenVIS introduces a query-centric framework for strengthening image-trained VIS. During single-frame training, we enrich Mask2Former queries with two auxiliary heads: a feature-prediction loss that aligns each query with the pooled backbone descriptor of its instance, and a center-prediction loss that injects spatial structure. Both heads are discarded at inference, adding zero parameters, and temporal identity is maintained by a training-free query-propagation and memory-bank scheme. On YouTube-VIS and OVIS with a ResNet-50 backbone, QueenVIS improves over MinVIS, up to +6.7 AP on YouTube-VIS, +4.8 AP on OVIS, and +10.3 AP on the long-sequence YouTube-VIS split. QueenVIS achieves 50.9 AP on YouTube-VIS and remains competitive with recent video-supervised state-of-the-art, without processing a single video clip during training. Our findings suggest that strengthening the discriminative power and temporal stability of object queries is an important, underexplored axis for VIS. Code and models: https://github.com/ArianKheir/QueenVIS