Omni-AD: A Large-scale and Versatile Benchmark for Industrial Anomaly Detection

Industrial Anomaly Detection (IAD) has attracted significant attention and witnessed rapid development. However, the advancement in this field is hindered by two key issues: the performance saturation of existing benchmarks, limiting discriminative evaluation of different IAD methods, and the absence of benchmarks tailored to assess recent multi-modal large language models (MLLMs) in anomaly detection. To this end, we present Omni-AD, a comprehensive IAD benchmark featuring: i) Large scale: The dataset consists of approximately 35K images (6× larger than MVTec AD) with 150 product categories (10× larger than MVTec AD) spanning 16 industrial sectors, delivering unprecedented diversity in terms of both category and image scale compared with existing datasets. ii) Versatility: The benchmark supports both conventional unsupervised and emerging MLLM-based IAD evaluation protocols. The latter is achieved by defining three subtasks of progressive difficulty, with two structured as visual question answering (VQA) and one as visual grounding. iii) Challenge: Extensive experimental results of state-of-the-art methods reveal that the Omni-AD benchmark is more challenging than existing benchmarks, which can drive the future development of the IAD field.