Search papers, labs, and topics across Lattice.
This paper introduces AnomalyAgent, a training-free agentic framework for zero- and few-shot anomaly detection that leverages multimodal large language models (MLLMs). The framework incorporates an anomaly-centric toolset for adaptive, MLLM-driven reasoning and a customized memory module for grounding reasoning with in-context examples. Experiments across diverse anomaly types, including logical and contextual anomalies, demonstrate AnomalyAgent's superior generalization compared to existing training-free VLM-based and generic agentic methods.
Forget training wheels: AnomalyAgent uses the reasoning power of multimodal LLMs to spot anomalies in zero- or few-shot settings, outperforming traditional VLM approaches.
Benefiting from generalizability of vision-language models (VLMs) such as CLIP, many zero-/few-shot anomaly detection (AD) approaches have achieved impressive detection performance across various datasets. Nevertheless, they require substantial training on large auxiliary datasets to adapt VLMs to anomaly detection, and their inference largely relies on visual-text embedding similarity-based anomaly scores, lacking reasoning abilities to detect complex anomalies that require in-depth contextual understanding. To address this limitation, we propose \textbf{AnomalyAgent}, a novel training-free, agentic framework that leverages the advanced reasoning and generalization capabilities of multimodal large language models (MLLMs) for anomaly detection. The key ingredients include \textbf{1)} a comprehensive anomaly-centric toolset that enables adaptive MLLM-driven, agentic anomaly reasoning in zero-shot settings, and \textbf{2)} a customized memory module that grounds anomaly reasoning with few-shot, in-context reference examples. We extend evaluation beyond the detection of simple anomalies (e.g., surface defects like cracks and dents and clear lesions) in widely used benchmarks to more diverse types of anomalies such as logical/contextual anomalies in logistics and manufacturing settings. Extensive experiment results demonstrate that our AnomalyAgent achieves substantially better performance compared to training-free VLM-based AD and generic agentic methods, highlighting its superior generalization capability in both zero-shot and few-shot anomaly detection settings. The code implementation can be find at this address.
