Search papers, labs, and topics across Lattice.
VisionPangu, a 1.7B-parameter multimodal model, is introduced to enhance detailed image captioning by focusing on efficient multimodal alignment and high-quality supervision. It leverages an InternVL-derived vision encoder and OpenPangu-Embedded language backbone, connected by a lightweight MLP projector, and is trained using an instruction-tuning pipeline inspired by LLaVA. By incorporating dense human-authored descriptions from the DOCCI dataset, VisionPangu achieves competitive performance with more structured and detailed captions compared to larger models.
A 1.7B-parameter model can rival larger LMMs in detailed image captioning, proving that high-quality supervision and efficient architecture are more crucial than brute-force scaling.
Large Multimodal Models (LMMs) have achieved strong performance in vision-language understanding, yet many existing approaches rely on large-scale architectures and coarse supervision, which limits their ability to generate detailed image captions. In this work, we present VisionPangu, a compact 1.7B-parameter multimodal model designed to improve detailed image captioning through efficient multimodal alignment and high-quality supervision. Our model combines an InternVL-derived vision encoder with the OpenPangu-Embedded language backbone via a lightweight MLP projector and adopts an instruction-tuning pipeline inspired by LLaVA. By incorporating dense human-authored descriptions from the DOCCI dataset, VisionPangu improves semantic coherence and descriptive richness without relying on aggressive model scaling. Experimental results demonstrate that compact multimodal models can achieve competitive performance while producing more structured and detailed captions. The code and model weights will be publicly available at https://www.modelscope.cn/models/asdfgh007/visionpangu.
