Search papers, labs, and topics across Lattice.
The paper introduces a multimodal AI framework for maritime multi-scene recognition that fuses image data, textual descriptions, and MLLM-generated classification vectors to improve recognition accuracy in complex marine environments. They employ an efficient multimodal fusion mechanism to enhance model robustness. The proposed model achieves 98% accuracy, surpassing previous state-of-the-art models by 3.5%, and is optimized for resource-constrained platforms using activation-aware weight quantization (AWQ), reducing the model size to 68.75MB with a minimal accuracy drop.
Achieve 98% accuracy in maritime scene recognition by fusing image data, text descriptions, and MLLM outputs, then compress the model to 68MB with minimal performance loss.
Maritime Multi-Scene Recognition is crucial for enhancing the capabilities of intelligent marine robotics, particularly in applications such as marine conservation, environmental monitoring, and disaster response. However, this task presents significant challenges due to environmental interference, where marine conditions degrade image quality, and the complexity of maritime scenes, which requires deeper reasoning for accurate recognition. Pure vision models alone are insufficient to address these issues. To overcome these limitations, we propose a novel multimodal Artificial Intelligence (AI) framework that integrates image data, textual descriptions and classification vectors generated by a Multimodal Large Language Model (MLLM), to provide richer semantic understanding and improve recognition accuracy. Our framework employs an efficient multimodal fusion mechanism to further enhance model robustness and adaptability in complex maritime environments. Experimental results show that our model achieves 98$\%$ accuracy, surpassing previous SOTA models by 3.5$\%$. To optimize deployment on resource-constrained platforms, we adopt activation-aware weight quantization (AWQ) as a lightweight technique, reducing the model size to 68.75MB with only a 0.5$\%$ accuracy drop while significantly lowering computational overhead. This work provides a high-performance solution for real-time maritime scene recognition, enabling Autonomous Surface Vehicles (ASVs) to support environmental monitoring and disaster response in resource-limited settings.
