Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a novel texture-aware transformer network for semantic segmentation of textured 3D meshes, addressing the limitations of existing methods that often neglect textural information. The core innovation is a texture branch that summarizes face-level pixels into a learnable token, fused with geometric descriptors and processed by Two-Stage Transformer Blocks for local and global information flow. Experiments on the SUM benchmark and a new cultural-heritage dataset demonstrate significant performance gains, achieving 81.9% mF1 and 94.3% OA on SUM.
By explicitly incorporating textural information into a transformer-based architecture, this approach achieves state-of-the-art semantic segmentation on textured 3D meshes, unlocking more accurate scene understanding from complex geometric data.
Textured 3D meshes jointly represent geometry, topology, and appearance, yet their irregular structure poses significant challenges for deep-learning-based semantic segmentation. While a few recent methods operate directly on meshes without imposing geometric constraints, they typically overlook the rich textural information also provided by such meshes. We introduce a texture-aware transformer that learns directly from raw pixels associated with each mesh face, coupled with a new hierarchical learning scheme for multi-scale feature aggregation. A texture branch summarizes all face-level pixels into a learnable token, which is fused with geometrical descriptors and processed by a stack of Two-Stage Transformer Blocks (TSTB), which allow for both a local and a global information flow. We evaluate our model on the Semantic Urban Meshes (SUM) benchmark and a newly curated cultural-heritage dataset comprising textured roof tiles with triangle-level annotations for damage types. Our method achieves 81.9\% mF1 and 94.3\% OA on SUM and 49.7\% mF1 and 72.8\% OA on the new dataset, substantially outperforming existing approaches.