Search papers, labs, and topics across Lattice.
This paper analyzes sprite-based image decomposition models, identifying their core components and limitations in scalability and dataset specificity. The authors leverage this analysis to propose a deep sprite-based image decomposition method that scales linearly with the number of objects and explicitly identifies object categories. The proposed method achieves state-of-the-art performance on the CLEVR benchmark for unsupervised class-aware image segmentation while maintaining high interpretability.
Sprite-based image models, long overlooked, can now achieve state-of-the-art unsupervised segmentation with linear scaling, thanks to a deep learning approach.
While foundation models drive steady progress in image segmentation and diffusion algorithms compose always more realistic images, the seemingly simple problem of identifying recurrent patterns in a collection of images remains very much open. In this paper, we focus on sprite-based image decomposition models, which have shown some promise for clustering and image decomposition and are appealing because of their high interpretability. These models come in different flavors, need to be tailored to specific datasets, and struggle to scale to images with many objects. We dive into the details of their design, identify their core components, and perform an extensive analysis on clustering benchmarks. We leverage this analysis to propose a deep sprite-based image decomposition method that performs on par with state-of-the-art unsupervised class-aware image segmentation methods on the standard CLEVR benchmark, scales linearly with the number of objects, identifies explicitly object categories, and fully models images in an easily interpretable way.
