Anomaly Detection by Adapting a Pre-Trained Vision Language Model

Recently, large vision and language models have shown their success whenadapting them to many downstream tasks. In this paper, we present a unifiedframework named CLIP-ADA for Anomaly Detection by Adapting a pre-trained CLIPmodel. To this end, we make two important improvements: 1) To acquire unifiedanomaly detection across industrial images of multiple categories, we introducethe learnable prompt and propose to associate it with abnormal patterns throughself-supervised learning. 2) To fully exploit the representation power of CLIP,we introduce an anomaly region refinement strategy to refine the localizationquality. During testing, the anomalies are localized by directly calculatingthe similarity between the representation of the learnable prompt and theimage. Comprehensive experiments demonstrate the superiority of our framework,e.g., we achieve the state-of-the-art 97.5/55.6 and 89.3/33.1 on MVTec-AD andVisA for anomaly detection and localization. In addition, the proposed methodalso achieves encouraging performance with marginal training data, which ismore challenging.