Search papers, labs, and topics across Lattice.
AMALIA-VL is the first open-source instruction-tuned vision and language model specifically designed for European Portuguese, addressing the lack of representation in existing multimodal models. The model employs a three-stage training process that includes vision-language alignment, general visual instruction tuning, and preference optimization, utilizing a unique pt-PT-centric multimodal dataset. Evaluation results indicate that AMALIA-VL sets a strong baseline for future development of LVLMs in European Portuguese, with plans to release resources to foster further research in this area.
European Portuguese finally gets its own dedicated vision and language model, setting a new standard for inclusivity in multimodal AI.
Large Vision and Language Models (LVLMs) have advanced rapidly, yet European Portuguese (pt-PT) remains systematically underserved by existing open-source multimodal models, which either conflate it with Brazilian Portuguese or severely under-represent it in their training data mixes. We introduce AMALIA-VL, the first open-source instruction-tuned LVLM built natively for pt-PT, pairing a high-resolution vision encoder with dynamic image tiling and a fully open pt-PT-optimized language model via a learned connector. We contribute with a purposefully designed three-stage training process - vision-language alignment, general visual instruction tuning, and preference optimization - together with a pt-PT-centric multimodal data mix combining curated and translated public datasets with novel datasets that address the near-total absence of European Portuguese multimodal resources. Our evaluation shows that AMALIA-VL establishes a strong baseline for open-source pt-PT LVLMs.We will release model weights, training data, and construction pipelines along with machine-translated pt-PT evaluation benchmarks to help democratize pt-PT LVLM development.
