Search papers, labs, and topics across Lattice.
The paper introduces EAGLE, a tuning-free framework for industrial anomaly detection that leverages expert model outputs to guide multimodal large language models (MLLMs) towards improved accuracy and interpretable anomaly descriptions. EAGLE uses expert model predictions to augment the attention mechanisms within MLLMs, focusing their attention on anomalous image regions. Experiments on MVTec-AD and VisA datasets demonstrate that EAGLE enhances anomaly detection performance across various MLLMs without requiring parameter updates, achieving results comparable to fine-tuned models.
Skip the fine-tuning: expert guidance unlocks MLLMs for industrial anomaly detection, rivaling fine-tuned performance without the parameter updates.
Industrial anomaly detection is important for smart manufacturing, but many deep learning approaches produce only binary decisions and provide limited semantic explanations. Multimodal large language models (MLLMs) can potentially generate fine-grained, language-based analyses, yet existing methods often require costly fine-tuning and do not consistently improve anomaly detection accuracy compared to lightweight specialist detectors. We propose expert-augmented attention guidance for industrial anomaly detection in MLLMs (EAGLE), a tuning-free framework that integrates outputs from expert model to guide MLLMs toward both accurate detection and interpretable anomaly descriptions. We further study how EAGLE affects MLLMs internals by examining the attention distribution of MLLMs to the anomalous image regions in the intermediate layers. We observe that successful anomaly detection is associated with increased attention concentration on anomalous regions, and EAGLE tends to encourage this alignment. Experiments on MVTec-AD and VisA show that EAGLE improves anomaly detection performance across multiple MLLMs without any parameter updates, achieving results comparable to fine-tuning based methods. Code is available at \href{https://github.com/shengtun/Eagle}{https://github.com/shengtun/Eagle}
