Search papers, labs, and topics across Lattice.
This paper introduces CIAN, a multi-stage framework for event-enriched image captioning that integrates external narratives to enhance traditional captioning methods. By leveraging SigLIP for article retrieval, a LoRA-fine-tuned Qwen model for narrative generation, and N-Gram-based refinement, CIAN significantly improves caption quality on the OpenEvents-V1 benchmark. The results demonstrate a remarkable increase in CIDEr scores, indicating that retrieval-augmented generation can produce more contextually aware and coherent captions than existing pixel-based models.
CIAN boosts image captioning quality by integrating external narratives, achieving a CIDEr score increase from 0.030 to 0.094 on the OpenEvents-V1 benchmark.
Event-enriched image captioning describes not only visible content but also the broader context of events, including timing, location, and participants, capabilities missing in most pixel-bound models. We propose the Contextual Image-Article Narrator (CIAN), a multi-stage framework that enriches captions with external narratives. CIAN retrieves relevant articles using SigLIP, summarizes them to guide a Narrative Generation stage with a LoRA-fine-tuned Qwen model, and applies N-Gram-based Refinement for fluency and coherence. On the OpenEvents-V1 benchmark, CIAN achieves high retrieval performance (mAP 0.979) and improves caption quality, increasing CIDEr from 0.030 to 0.094. These results highlight the effectiveness of retrieval-augmented reasoning combined with linguistic refinement for generating context-aware, human-like captions.
