Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
Slimmable ConvNeXt enables width-adaptive inference for ConvNeXt models, allowing deployment across devices with varying resource constraints without training separate models. By leveraging ConvNeXt's LayerNorm and inverted bottlenecks, they eliminate the need for switchable batch normalization, simplifying the training pipeline compared to previous CNN and ViT approaches. Experiments on ImageNet-1k demonstrate that Slimmable ConvNeXt achieves superior accuracy and efficiency compared to existing width-adaptive models like HydraViT, MatFormer, and SortedNet.
ConvNeXt's modern design makes it surprisingly amenable to channel-width slimming, enabling efficient multi-device deployment without the normalization overhead of previous approaches.
Deploying vision models across devices with varying resource constraints, or even on a single device where available compute fluctuates due to battery state, thermal throttling, or latency deadlines, typically requires training and maintaining separate models. Width-adaptive inference addresses this by training a single set of shared weights containing multiple nested subnetworks of increasing capacity, but prior CNN-based approaches required switchable batch normalization, while recent scalable methods have focused on Vision Transformers. We present Slimmable ConvNeXt, which shows that ConvNeXt's modern design, specifically LayerNorm and inverted bottlenecks, makes it particularly suited for channel-width slimming, eliminating the normalization overhead of classical slimmable networks and producing a simpler training pipeline than both prior CNN and ViT approaches. On ImageNet-1k, Slimmable ConvNeXt-T with 3 subnetworks achieves 80.8% top-1 accuracy at 4.5 GMACs and 77.4% at 1.2 GMACs, trained from scratch for 600 epochs. At comparable compute, this exceeds HydraViT's 6-head subnetwork (78.4% at 4.6 GMACs) by 2.4 percentage points and its 3-head configuration (73.0% at 1.3 GMACs) by 4.4 percentage points, while also outperforming MatFormer-S (78.6%) and SortedNet-S (78.2%) at the same GMACs. Scaling to Slimmable ConvNeXt-B further improves maximum accuracy to 82.8% at 15.35 GMACs.