Search papers, labs, and topics across Lattice.
This paper introduces SPAR, a unified multimodal framework that addresses the challenges of aligning semantic perception with pixel-level reconstruction in Multimodal Large Language Models (MLLMs). By employing an asymmetric dual-stream tokenizer and a self-aligned generation paradigm, SPAR enhances the fidelity of visual generation without relying on external teachers. Experimental results show that SPAR achieves state-of-the-art performance in both generation and reconstruction tasks while maintaining strong visual understanding capabilities.
SPAR bridges the critical gap between semantic perception and pixel-level generation, achieving unprecedented quality in visual outputs without external supervision.
Multimodal Large Language Models (MLLMs) have achieved remarkable success in visual understanding but remain constrained in visual generation due to the fundamental feature discrepancy between semantic perception and pixel-level reconstruction. Bridging this gap requires overcoming two core challenges: endowing semantic encoders with high-fidelity reconstruction capabilities, and effectively aligning generative models with semantic spaces without relying on external teachers. To this end, we propose a novel unified multimodal framework featuring \textbf{S}emantic-\textbf{P}ixel self-alignment and \textbf{A}daptive \textbf{R}outing (\textbf{SPAR}). First, to reconcile semantic perception with pixel-level reconstruction, we introduce an asymmetric dual-stream unified tokenizer. A lightweight semantic stream anchors discriminative features, while a Transformer-augmented pixel stream recovers fine-grained visual details into a unified compact latent space. Second, to eliminate external dependencies, we propose a self-aligned generation paradigm that natively leverages this optimized tokenizer as an internal alignment teacher for the diffusion model. Furthermore, to facilitate flexible multimodal interaction within this unified space, we introduce Dynamic Token Routing, which enables each token to adaptively aggregate multi-layer MLLM features based on its distinct semantic demands. Extensive experiments demonstrate that SPAR establishes the state-of-the-art for unified architectures, achieving exceptional generation and reconstruction quality while preserving foundational visual understanding capabilities.
