Search papers, labs, and topics across Lattice.
MIT CSAILFeb 27, 2025This paper introduces a VQA system leveraging Visual BERT, ViLT, cross-modal memory networks, memory-augmented attention, and vision-language pre-training models (Flamingo, BLIP) for improved multimodal fusion and dynamic memory retrieval. The system addresses complex reasoning by adapting to novel question types through few-shot learning. Experiments on VQA v2.0 demonstrate 80% accuracy, surpassing LSTM-CNN and attention-only baselines, alongside improved BLEU scores and precision-recall metrics.
Achieving 80% accuracy on VQA v2.0 proves that combining Visual BERT, ViLT, and memory-augmented attention can significantly outperform traditional VQA models.
Visual Question Answering is a challenging task requiring the integration of visual and textual information to answer questions about images. This research introduces a robust VQA system that combines advanced deep learning techniques, including Visual BERT, ViLT, cross-modal memory networks, memory-augmented attention mechanisms, and vision language pre-training models such as Flamingo and BLIP. These methods enable effective multimodal fusion, dynamic memory retrieval, and adaptation to novel question types using few-shot learning. Using the VQA v2.0 dataset, the system achieved 80% accuracy, outperforming traditional models such as LSTM-CNN and attention-only approaches. Enhanced BLEU scores and precision-recall metrics underscore its capability in handling complex, multi-step reasoning tasks. This work highlights significant improvements in VQA systems and establishes a scalable framework for advancing multimodal learning, paving the way for future research on larger datasets and real-world applications.
