Search papers, labs, and topics across Lattice.
This paper introduces the Dynamic Dual-level Vision Transformer Fusion Network (DDVT) for enhancing answer grounding in visual question answering (VQA) by effectively localizing visual content associated with natural language questions. The core innovation lies in the question-guided dynamic regional-level module (QGDR) that integrates image context and text content through ROI Align, alongside a cross-modal multi-scale aggregation module (CMA) that optimizes feature fusion across different scales. Experimental results indicate that DDVT significantly surpasses existing state-of-the-art methods on multiple benchmarks, highlighting its effectiveness in precise localization tasks.
DDVT achieves superior answer grounding in visual question answering by seamlessly integrating visual and textual features, outperforming existing methods on key benchmarks.
Answer grounding in visual question answering aims to locate the region from a given natural language question associated with the visual content of an image, which has garnered significant attention due to its practical applications. In this paper, we introduce the Dynamic Dual-level Vision Transformer Fusion Network (DDVT) for answer grounding in visual question answering. Specifically, we propose a question-guided dynamic regional-level module (QGDR) that combines complementary image context through ROI Align and text content, enabling precise localization of text-related visual content. Moreover, we present a cross-modal multi-scale aggregation module (CMA) that enhances feature fusion between pixel-level and region-level features, facilitating the effective localization of visual content associated with grounded answers. Furthermore, we fuse the located visual content with text features to locate the region and provide answers to questions posed about the image. Experimental results demonstrate that our DDVT outperforms state-of-the-art methods on several widely-used benchmarks.
