Track the frontier, from condensed matter to cosmology.
Search papers, labs, and topics across Lattice.
Xiaomi-Lens-0, a 4B MLLM, addresses fine-grained region understanding by predicting masks directly from its visual representations, bypassing the need for explicit text-region supervision. This is achieved by a lightweight decoder that fuses patch features with language model mask-token states, enabling the model to interleave reasoning with region-specific latents. The model demonstrates significant improvements in referring comprehension, achieving a 5.6-point average gain over its base model, and sets a new state-of-the-art in segmentation cIoU.
LLMs can now "think with visual latents" by generating masks directly from their internal representations, dramatically improving referring comprehension and segmentation.
Multimodal large language models (MLLMs) answer questions about whole images well, yet they still struggle with fine-grained region understanding. A key reason is that text-only supervision never specifies which regions support an answer. To close this gap, we present Xiaomi-Lens-0, a 4B MLLM built on Qwen3.5 that predicts masks from its own visual representations. Specifically, a lightweight decoder combines vision-encoder patch features with visual latents, the language model's mask-token states, so mask supervision reaches the shared backbone without an additional segmentation encoder. Because each latent decodes to a mask, the model can interleave reasoning with region-specific latents and thus think with visual latents, while grounded reasoning traces supervise the regions it visits during chain-of-thought. As a result, across all eight RefCOCO, RefCOCO+, and RefCOCOg splits, Xiaomi-Lens-0 gains 5.6 points on average over Qwen3.5 in referring comprehension and achieves the highest segmentation cIoU among compared models. Moreover, it surpasses Qwen3.5 across all perception benchmarks. Homepage: https://seerray-lab.github.io/Xiaomi-Lens-0{this https URL}