Search papers, labs, and topics across Lattice.
This paper introduces Hierarchical Slot Attention (HSA), a novel framework that enables multi-granularity semantic scene decomposition from a single model, addressing the limitations of existing slot attention methods that rely on flat, appearance-based decompositions. HSA operates at three levels鈥攈olistic, semantic, and panoptic鈥攁llowing for a more nuanced understanding of scenes akin to human perception, and achieves this with only 10% labeled data through hierarchical alignment loss. Experimental results on COCO and PASCAL VOC show significant improvements over the strongest flat baseline, with increases of up to 41.5 ARI at the holistic level, underscoring the effectiveness of hierarchical learning in object-centric tasks.
HSA achieves up to 41.5% improvement in scene decomposition accuracy by leveraging hierarchical semantic understanding with minimal labeled data.
Slot attention is a powerful framework for object-centric learning, decomposing visual scenes into latent slots through iterative competitive attention. However, existing methods share two critical limitations: they decompose scenes into a flat set of slots at a single granularity, and this decomposition is based on appearance rather than semantics. Yet humans understand scenes through semantic hierarchies: separating foreground from background, recognizing object categories, and identifying individual instances. Crucially, such semantic hierarchies cannot emerge without supervision, because category names are human constructs, not visual patterns. We propose Hierarchical Slot Attention (HSA), which learns multi-granularity semantic scene decomposition from a single model. HSA decomposes scenes at three levels: holistic (foreground/background), semantic (object categories), and panoptic (individual instances). Using only 10\% labeled data, combined with hierarchical alignment loss, HSA learns all three levels jointly. We further introduce grouping purity and containment to measure whether the hierarchy is encoded in representation space, not just output masks. Experiments on COCO and PASCAL VOC demonstrate that HSA outperforms the strongest flat baseline by up to \textbf{$+$41.5} ARI at holistic, \textbf{$+$14.6} at semantic, and \textbf{$+$10.4} at panoptic level on COCO, with even larger gains on Pascal VOC, while requiring a single model instead of three. Code will be made available upon acceptance.
