Search papers, labs, and topics across Lattice.
This study challenges the prevailing notion that Vision Transformers (ViTs) inherently require more labeled data than CNNs for industrial dense prediction by revealing that the true cause of the data-efficiency gap is pretraining incoherence between ViT backbones and CNN necks. Through controlled experiments across four industrial datasets, the authors introduce the AlignBlock family for recalibrating pyramid-level features, which significantly enhances performance. Key results show that for domain-proximal scenes, the Swin-Graft model outperforms YOLOv11x in mAP@50, while CNNs maintain their edge in domain-distant scenarios.
Pretraining incoherence, not self-attention deficits, explains why ViTs can outperform CNNs in low-data settings.
Vision Transformers (ViTs) are widely believed to require more labeled data than CNNs for industrial dense prediction. Through controlled experiments on four industrial datasets, we show that the data-efficiency gap stems from pretraining incoherence, which refers to the statistical mismatch between ImageNet-pretrained ViT backbones and COCO-pretrained CNN necks, rather than from inherent self-attention deficits. We characterize the cross-architecture feature gap and propose a lightweight AlignBlock family for pyramid-level feature recalibration. Our core finding empirically identifies a data-efficiency frontier: for domain-proximal scenes with >= 200 samples, Swin-Graft surpasses YOLOv11x (terminal 703-shot: 0.973 vs 0.956 mAP@50); for domain-distant scenes, CNNs retain advantage (hook 141-shot: 0.900 vs 0.600 mAP@50). Grafted neck weights yield up to 2.5x the mAP of a randomly initialized neck.
