Search papers, labs, and topics across Lattice.
The paper introduces LLandMark, a multi-agent framework for landmark-aware multimodal video retrieval, designed to handle complex queries by integrating domain-specific knowledge and adaptive reasoning. LLandMark employs specialized agents for query parsing, landmark reasoning (using CLIP and LLMs), multimodal retrieval, and answer synthesis, with a focus on Vietnamese cultural and spatial landmarks. The framework leverages Gemini 2.5 Flash for landmark detection and image-to-image retrieval, and enhances Vietnamese OCR using Gemini and LlamaIndex, demonstrating improved retrieval performance.
Forget manual image inputs: LLandMark uses Gemini 2.5 Flash to autonomously detect landmarks in videos and generate image search queries for enhanced multimodal retrieval.
The increasing diversity and scale of video data demand retrieval systems capable of multimodal understanding, adaptive reasoning, and domain-specific knowledge integration. This paper presents LLandMark, a modular multi-agent framework for landmark-aware multimodal video retrieval to handle real-world complex queries. The framework features specialized agents that collaborate across four stages: query parsing and planning, landmark reasoning, multimodal retrieval, and reranked answer synthesis. A key component, the Landmark Knowledge Agent, detects cultural or spatial landmarks and reformulates them into descriptive visual prompts, enhancing CLIP-based semantic matching for Vietnamese scenes. To expand capabilities, we introduce an LLM-assisted image-to-image pipeline, where a large language model (Gemini 2.5 Flash) autonomously detects landmarks, generates image search queries, retrieves representative images, and performs CLIP-based visual similarity matching, removing the need for manual image input. In addition, an OCR refinement module leveraging Gemini and LlamaIndex improves Vietnamese text recognition. Experimental results show that LLandMark achieves adaptive, culturally grounded, and explainable retrieval performance.
