Search papers, labs, and topics across Lattice.
This paper investigates the design principles crucial for effective pixel text representation learning, addressing limitations of existing encoders such as fixed resolution pretraining and weak visual grounding. Through systematic ablation studies, the authors identify four key components that enhance model performance, including variable image resolutions, natural image-text pairs, layout-aware rendering, and a two-stage multilingual curriculum. The resulting model, Pixel Linguist II, achieves state-of-the-art results across multiple benchmarks while demonstrating robustness to significant visual token compression.
Pixel Linguist II not only sets new benchmarks in multilingual visual text understanding but also maintains performance under 80% visual token compression.
Text-rich visual inputs require models that can read, retrieve, and compress language directly in pixel space, yet existing pixel-text encoders struggle with fixed resolution pretraining, visual shortcut learning, weak visual grounding, and multilingual visual text understanding. In this work, we investigate the fundamental design principles required for robust visual text representation learning. Through systematic controlled ablations, we identify four critical components: variable image resolutions and rendered font sizes provide spatial proxies for high-resolution document generalization; natural image-text pairs are indispensable for grounding and prevent text-only collapse; layout-aware rendering helps prevent pixel-level shortcuts; and a two-stage multilingual curriculum enables effective cross-lingual alignment. By integrating these principles into a scalable training recipe, we train Pixel Linguist II, a native-resolution vision encoder trained with on-the-fly rendering, unified contrastive grounding, and a multilingual curriculum over 280M training examples. Pixel Linguist II sets new state-of-the-art results on English, cross-lingual, and multilingual Visual STS and ViDoRe, while also enabling better MLLM downstream evaluation. Notably, Pixel Linguist II remains robust under 80\% visual token compression, showing great promise for optical context compression. Our code and resources are available at https://github.com/Pixel-Linguist/Pixel-Linguist-II.
