Search papers, labs, and topics across Lattice.
This paper introduces ARMDIL, an Adaptive Router for Multi-Domain Image classification that utilizes a multimodal large language model (MLLM) to dynamically route images to the most appropriate vision backbone from a diverse ensemble of architectures. By leveraging convolutional neural networks, self-supervised learners, and vision-language models, ARMDIL effectively addresses the challenges of generalization across varying datasets and domains. The results demonstrate that ARMDIL not only competes with specialized routers but also enhances adaptability and interpretability through simple prompt modifications and natural language reasoning traces.
ARMDIL outperforms traditional routers by dynamically selecting the best-suited model for each image, significantly enhancing cross-domain generalization.
Modern image classification models excel when trained on single task-specific datasets but often struggle to generalize across domains and difficulty levels. We propose ARMDIL, an Adaptive Router for Multi-Domain Image classification with LLMs. ARMDIL is an ensemble that uses a multimodal large language model (MLLM) agent to dynamically route each image to the most suitable vision backbone. Our diverse ensemble employs convolutional neural networks (ResNets), self-supervised representation learners (SSL), and vision-language models (VLMs), each trained on a unified label space constructed from multiple image datasets with differing distributions and characteristics. Empirical evaluations illuminate the distinct capabilities and vulnerabilities of each architecture across disparate visual domains. Crucially, we show that ARMDIL effectively navigates these trade-offs, performing competitively with specialized training-based routers. Furthermore, it drastically improves adaptability by allowing new information to be integrated via simple prompt modifications, while enhancing interpretability through natural language reasoning traces. These advances in cross-dataset image classification pave the way for more reliable general-purpose vision systems such as AI assistants and autonomous robots.
