Search papers, labs, and topics across Lattice.
This paper introduces Single-Patch Text Spotting (SPaTS), a novel framework for scene text spotting that optimizes visual-token grounding by utilizing a single anchor visual token for each text instance, thereby reducing noise and ambiguity. The authors employ a reinforcement learning approach called Single-Patch Selective Optimization (SPaSO) to enhance the selection of visual tokens without relying on oracle labels, while also implementing Directional Embedding Alignment (DEA) and Patch-Enhanced Decoding (PED) to improve localization precision and representation robustness. Experimental results show that SPaTS significantly outperforms existing state-of-the-art models in both closed-source and OCR contexts, highlighting its effectiveness in practical applications.
Single-Patch Text Spotting achieves unprecedented accuracy in scene text spotting by leveraging a single visual token per instance, outperforming leading models in the field.
Scene text spotting requires high-precision alignment between textual recognition and spatial localization. While visual-token grounding has emerged as a promising formulation for Multimodal Large Language Models (MLLMs), the previous multi-patch paradigm often introduces redundant noise and localization ambiguity, particularly for dense or small text instances. To address this, we propose Single-Patch Text Spotting (SPaTS), a vision-centric framework that routes each text instance through a single anchor visual token and then recovers geometry via full-image refinement. To accurately identify this anchor without oracle labels, we introduce Single-Patch Selective Optimization (SPaSO), a reinforcement learning framework that optimizes discrete visual-token selection using patch-level rewards. To further improve representation robustness and localization precision, we introduce Directional Embedding Alignment (DEA) to suppress unstable norm bias by decoupling feature magnitude and direction, and Patch-Enhanced Decoding (PED) to fuse the routed anchor with language semantics and cross-attend over the full-image feature map for geometry-aware boundary regression beyond coordinate-space surrogates. Extensive experiments demonstrate that SPaTS consistently and significantly outperforms both frontier closed-source MLLMs and OCR MLLMs. Code will be released soon.
