Search papers, labs, and topics across Lattice.
This paper introduces Semantic Prism, a novel framework for generative semantic segmentation that addresses issues of color drift and boundary mixing in traditional methods by employing a conditional semantic-image generation-and-refinement approach. By utilizing a diffusion-distilled generator and Hierarchical Generator Evidence Alignment, the framework achieves significant improvements in segmentation accuracy, evidenced by a mean intersection over union (mIoU) of 72.07% on the Cityscapes dataset, surpassing previous methods by 11.39 mIoU points. Additionally, the Contextual Interface--Hierarchy Disagreement (C-IHD) technique enhances pixel-error ranking, leading to improved precision-recall metrics across multiple datasets.
Semantic Prism outperforms traditional segmentation methods by over 11 mIoU points, demonstrating that hierarchical feature alignment can drastically enhance generative semantic segmentation performance.
Generative semantic segmentation exposes structured predictions as images, but direct color decoding is susceptible to color drift and boundary mixing, whereas latent-feature decoders that predict a separate output distribution may relegate the rendered image to an intermediate visualization. We present Semantic Prism, a conditional semantic-image generation-and-refinement framework with deterministic inference. A diffusion-distilled one-step generator renders a semantic RGB image; per-pixel distances from the rendered colors to a fixed class-color codebook define an explicit probabilistic interface. Hierarchical Generator Evidence Alignment spatially aligns multi-level generator features and uses a zero-initialized output projection to predict an additive residual in the interface logit space, retaining the image-defined interface as the reference for the final distribution. The interface and refined distributions further enable Contextual Interface--Hierarchy Disagreement (C-IHD), a fixed readout for ranking remaining pixel errors without an auxiliary predictor or additional forward pass. On the 500-image Cityscapes validation set, Semantic Prism achieves 72.07% mean intersection over union, 11.39 mIoU points above direct-interface decoding, with 0.41% expected calibration error. Matched-capacity ablations over three seeds support the benefit of jointly aligned multi-level evidence. A separately trained model attains 62.22% mIoU on BDD100K, while the Cityscapes-trained model reaches 46.89\% mIoU under source-frozen transfer to the Adverse Conditions Dataset with Correspondences, without target-domain adaptation. Across all three datasets, C-IHD consistently improves the area under the precision--recall curve for pixel-error ranking over maximum softmax probability on the same segmentation predictions; on ACDC, it raises AUPR from 0.6580 to 0.7557.
