Search papers, labs, and topics across Lattice.
This paper details the re-implementation of the NAVER LABS instruction-following pipeline for the IWSLT 2026 Shared Task, utilizing SeamlessM4T-v2-large as the speech encoder and Qwen3-4B-Instruct as the LLM backbone. The authors maintain a three-stage approach involving projector alignment, text-only LoRA pre-training, and multimodal merging, while also generating 100k synthetic instruction-following examples across ten task types for enhanced fine-tuning. The resulting model achieves notable performance metrics, including a COMET score of 0.781 for EN-ZH speech translation and a BERTScore-F1 of 0.346 on the MCIF benchmark for English SQA.
Achieving a COMET score of 0.781 in EN-ZH speech translation highlights the effectiveness of synthetic data in enhancing instruction-following capabilities in multimodal models.
We re-implement the NAVER LABS IWSLT 2025 instruction-following pipeline for the IWSLT 2026 Shared Task (constrained condition, short audio track), adapting it to the mandated components: SeamlessM4T-v2-large as the speech encoder and Qwen3-4B-Instruct as the LLM backbone. The three-stage approach projector alignment, text-only LoRA pre-training, and multimodal merging is preserved from the original design. We additionally construct 100k synthetic instruction-following examples across ten speech-centric task types (10k per task) from the provided corpora, suitable for further Stage 3 fine-tuning. Our primary model achieves COMET 0.781 on EN-ZH speech translation and BERTScore-F1 0.346 on English SQA on the MCIF benchmark.
