Search papers, labs, and topics across Lattice.
This paper introduces the Multi-Resolution Pyramid Transformer (MRPT), a novel model designed to effectively process Gigapixel Whole Slide Images (WSIs) by hierarchically aggregating information across multiple resolutions. By employing a Consecutive Cross-Resolution Attention (CCRA) mechanism, MRPT captures scale-independent interactions and ensures semantic consistency across different resolutions, which is crucial for accurate pathology analysis. Extensive evaluations demonstrate that MRPT significantly outperforms existing foundation models and Multimodal Large Language Models (MLLMs) in tasks such as cancer subtype classification and Visual Question Answering, highlighting its robustness and generalizability in computational pathology.
MRPT's innovative approach to multi-resolution attention enables it to outperform state-of-the-art models in cancer classification and tissue analysis, revealing new depths in diagnostic accuracy.
Vision Transformers (ViTs) and their hierarchical variants have achieved strong performance in Computational Pathology (CPath). However, most are pre-trained on single-resolution Whole Slide Images (WSIs), limiting their generalization across arbitrary resolutions. Gigapixel WSIs inherently contain diagnostic patterns at multiple scales, including cellular morphologies, tissue architectures, and global context, mirroring how expert pathologists examine WSIs. We introduce Multi-Resolution Pyramid Transformer (MRPT), a model that hierarchically aggregates multi-resolution information from cellular to tissue and WSI levels. MRPT employs a biologically meaningful Consecutive Cross-Resolution Attention (CCRA) mechanism to capture scale-independent interactions and enforces multi-resolution semantic consistency by aligning embeddings across resolutions, yielding robust and generalizable WSI representations. Pre-trained in a multi-resolution self-supervised manner on 624M patches, 2.4M regions, and 36K WSIs, MRPT learns rich coarse-to-fine histopathology features. Extensive experiments on 34 diverse datasets show that MRPT surpasses recent foundation models and Multimodal Large Language Models (MLLMs) in cancer subtype classification, tissue phenotyping, and Visual Question Answering (VQA) for WSI understanding.
