Visual Anomaly Detection

Visual anomaly detection decides whether an image, or a region of it, departs from what a set of normal images shows, when examples of the departures are absent or too few to learn from. A model is fitted to normal images only and then scores test images: one number per image for anomaly detection at image level, and one number per pixel, an anomaly map, for anomaly localization or segmentation at pixel level. The same problem appears under several names: novelty detection, outlier detection and one-class classification (Bergmann et al., 2021, list them as used more or less equivalently), “cold-start” anomaly detection (Roth et al., 2022), and, confusingly, both “unsupervised” (the MVTec papers, since no labels are used) and “semi-supervised” (Rudolph et al., 2021, since the training images are all known to be normal). The task’s main use is manufacturing inspection, where defects are rare and new kinds keep appearing; Industrial Visual Inspection covers that application, its imaging and its operating thresholds, while this article covers the task itself, its method families and its benchmarks.

Problem Definition

Training data are a set $X_N$ of normal images; nothing about the anomalies is given. At test time each image $x$ receives a score $s(x)$, larger meaning more anomalous, and for localization each pixel $p$ of image $i$ receives a score $A_i(p)$ (the notation of Bergmann et al., 2021, Section 4). A binary decision needs a threshold, which must also be chosen without anomalous examples. Bergmann et al. (2021, Section 5) argue that even when a few defective images exist, a threshold estimated from normal data alone is preferable, because the few defects cannot cover the range of anomalies that may occur.

Novelty detection and defect detection. Two settings share the name. In what Bergmann et al. (2021, Section 1) call novelty detection, the outliers differ strongly from the inliers at image level. Its standard protocol relabels classes of a classification dataset: Ruff et al. (2018) build ten one-class problems from MNIST and ten from CIFAR-10, each training only on one class’s training images and testing on the full test set, in which the nine other classes are the anomalies, scored by the area under the ROC curve (AUC). In anomaly detection in Bergmann et al.’s sense, which industrial defect detection exemplifies, anomalies are subtle deviations, possibly in very small regions, of images that otherwise look like the training data, and they should be segmented. Rudolph et al. (2021) make the same distinction, calling defect detection the sub-problem with visually similar normal samples and only slightly different anomalies. Results on one setting need not carry over: Bergmann et al. (2021, Section 2.1.1) point out that on relabelled classification datasets it is unclear how a method would do when anomalies are slight deviations from the training data.

Defect detection with labels. When defective examples are available in quantity, the problem becomes ordinary supervised classification or segmentation of defects, which Industrial Visual Inspection describes. Zou et al. (2022) define two-class protocols on VisA for that case; the anomaly detection setting is the one-class protocol, in which the training set holds normal images only.

Input and Output

The input is an image of the kind the model was fitted to: one product, one texture or one scene, usually under fixed imaging. Benchmarks train a separate model per category (Gudovskiy et al., 2022, Section 5.1). The outputs are:

• an image-level anomaly score, often the maximum of the anomaly map (PaDiM; Defard et al., 2021, Section III-C) or a re-weighted maximum over patches (PatchCore; Roth et al., 2022, Section 3.3);
• an anomaly map at the input resolution, usually upsampled from a coarser feature grid and smoothed (a Gaussian of $\sigma = 4$ pixels at 224 × 224 input in PaDiM and PatchCore);
• after thresholding, a binary decision and a defect mask.

Variants

• Image-level only. Some methods produce only an image score; DifferNet (Rudolph et al., 2021) was designed for detection and localizes defects only as a by-product, through gradients.
• Structural and logical anomalies. Scratches, dents and contaminations are local, structural anomalies. Logical anomalies violate underlying constraints, such as an allowed object in an invalid place or a required object missing; MVTec LOCO AD was built to include both (official page, as of November 2026).
• Few normal images. Rudolph et al. (2021) report DifferNet trained on only 16 images per category, and Roth et al. (2022, Section 4.5) evaluate PatchCore with 1 to 50 normal training images.
• Multiple instances and pose variation. VisA includes subsets with several objects per image and objects in widely varying positions (Zou et al., 2022).
• Video frames. PaDiM, PatchCore and CFLOW-AD also report localization on frames of the ShanghaiTech Campus dataset, a pedestrian-scene video benchmark whose test videos contain unusual events (Defard et al., 2021, Section V-B; Roth et al., 2022, Section 4.6; Gudovskiy et al., 2022, Table 5); Defard et al. compare only with methods that use no temporal information.

Challenges

• No anomalies to learn from. The model must describe normality well enough that anything else stands out, including defect types nobody anticipated (Bergmann et al., 2021, Section 1).
• Subtle and small anomalies. Defect areas in MVTec AD vary so widely that Bergmann et al. (2021, Fig. 3) plot them on a logarithmic scale.
• Normal variation. Wood grain, hazelnuts or deformable cables vary from one good sample to the next, and the model must accept that variation without accepting the defects.
• Alignment. Methods that model each image position separately assume aligned images; methods that compare patches regardless of position depend on alignment less (Roth et al., 2022, Section 2).
• Global context. Local patch models miss anomalies that are only visible in context. In the MVTec AD benchmark, the student–teacher method missed a transistor’s missing leg and the CNN feature dictionary a cable whose inner insulation had the wrong colour (Bergmann et al., 2021, Section 6.2).

Approaches

Defard et al. (2021) divide methods into reconstruction-based and embedding similarity-based families. Normalizing flows and student–teacher networks are further families that, like the embedding methods, start from pretrained features.

Reconstruction-based methods

An autoencoder or a generative adversarial network is trained to reconstruct normal images; at test time the difference between an image and its reconstruction is the anomaly map, on the assumption that the model cannot reproduce what it never saw. Schlegl et al.’s AnoGAN searches the GAN’s latent space for the sample that best reproduces the input, an optimization at every test image, and its successor f-AnoGAN adds an encoder for a single forward pass; Bergmann et al.’s SSIM autoencoder compares input and reconstruction by structural similarity instead of a per-pixel $\ell^2$ distance, to use local patch structure (all as described by Bergmann et al., 2021, Section 2.2).

These models localize poorly in both directions. In the MVTec AD benchmark (Bergmann et al., 2021, Section 6.2):

• they reconstruct fine detail blurrily, so anomaly scores rise on object boundaries, strong reflections and fine structure such as a toothbrush’s bristles, which are false positives;
• when a defect changes grey values only slightly, as with a transparent colour defect on the tile category, the per-pixel difference is small and the defect is missed;
• when the model “repairs” a structural defect in its reconstruction, the difference can still be small: f-AnoGAN missed a structural anomaly on carpet for this reason;
• the SSIM autoencoder works on grey-scale images and often failed to detect colour defects at all.

The opposite failure, a model that generalizes so well that it reconstructs anomalies too, is noted by Defard et al. (2021, Section II) and Rudolph et al. (2021, Section 2.1.2). Averaged over the 15 MVTec AD categories, the $\ell^2$ and SSIM autoencoders reached pixel-level ROC areas of 0.590 and 0.584 and f-AnoGAN 0.472, against 0.922 for the student–teacher method, all integrated up to a false-positive rate of 0.3 and normalized (Bergmann et al., 2021, Table 3). The history matters here: in the 2019 conference version, which already included the CNN feature dictionary on pretrained descriptors, the two autoencoders had done best on most object categories (Bergmann et al., 2019, Section 4.4), and only the 2021 benchmark, with updated implementations and the added student–teacher method, led the authors to conclude that methods using pretrained descriptors outperform generative models trained from scratch. Variational autoencoders were left out of that benchmark, because the literature they cite reported no significant gain over standard autoencoders and questioned the likelihoods of deep generative models (Section 2.2.2).

Embedding similarity on pretrained features

The family that took over describes each image patch by the activations of a network pretrained on ImageNet classification, without adapting it to the target data, and measures how far a test patch’s descriptor lies from the descriptors of normal patches. It is an application of transfer learning and of the idea, developed in representation learning, that a good representation makes simple distances meaningful. Early versions, as Bergmann et al. (2021, Section 2.2.3) describe them, clustered ResNet-18 descriptors of randomly cropped training patches with K-means (Napoletano et al.’s CNN feature dictionary, which needed heavy subsampling and a strided, slow evaluation), or fitted a single Gaussian to early AlexNet features (Sabokrou et al.), which cannot capture a complex feature distribution.

SPADE. Cohen and Hoshen’s SPADE (2020) keeps the features of all normal training images in a memory bank and scores test pixels by their distance to the nearest neighbours among the bank’s features from several network levels. As Roth et al. (2022, Section 3.2) describe it, the computational cost forces a preselection step: a weaker image-level stage on globally averaged last-layer features first selects which training feature maps are searched at pixel level. The SPADE paper, an arXiv preprint in the papers that cite it, could not be read here, so its results are given here as the papers that compared with it report them: an image-level AUROC of 85.5% on MVTec AD (Defard et al., 2021, Table V; Roth et al., 2022, Table 1) and a pixel-level AUROC of 96.5% according to Defard et al. (Table III) but 96.0% according to Roth et al. (Table 2) and Gudovskiy et al. (2022, Table 4). Defard et al. measured 7.10 s per image on a CPU (an Intel i7-4710HQ) for their reimplementation (Table VII), and attribute the slowness to the K-NN search over the stored training embeddings (Section V-D).

PaDiM. Defard et al. (2021) replace the memory bank with one multivariate Gaussian per patch position. Activations from the first three blocks of a ResNet are concatenated on the grid of the largest of the three maps, so each position $(i, j)$ of every training image yields an embedding $x_{ij}$; over the $N$ normal images, the mean $\mu_{ij}$ and covariance

$$\Sigma_{ij} = \frac{1}{N-1} \sum_{k=1}^{N} \left(x^k_{ij} - \mu_{ij}\right)\left(x^k_{ij} - \mu_{ij}\right)^\top + \epsilon I$$

are estimated, with $\epsilon = 0.01$ by default, and a test patch scores by its Mahalanobis distance $\sqrt{(x_{ij} - \mu_{ij})^\top \Sigma_{ij}^{-1} (x_{ij} - \mu_{ij})}$ (see similarity and distance measures); the image score is the maximum of the map (Section III). Two findings stand out. Modelling the correlations between network levels in one Gaussian beat summing separate per-level scores, by 1.1 points of pixel AUROC and 1.8 of PRO with ResNet-18 (Table I). And reducing the 448-dimensional embedding to 100 randomly chosen dimensions cost little (pixel AUROC 96.7% against 97.1%), whereas PCA to 100 dimensions cost much more (93.5%), which the authors explain by PCA keeping the high-variance directions, which need not be the ones that separate normal from anomalous (Table II). With a Wide ResNet-50 reduced to 550 dimensions, PaDiM reached a pixel AUROC of 97.5% and a PRO score of 92.1% on MVTec AD (Table III), and an image AUROC of 95.3%, or 97.9% with an EfficientNet-B5 backbone (Table V). Inference needs no search, so its cost does not grow with the training set. The price is the assumption of alignment, which the authors tested themselves with randomly rotated (±10°) and cropped copies of MVTec AD: pixel AUROC fell to 92.2%, still ahead of SPADE (87.2%) and a VAE (62.1%) on the same data (Table VI). Defard et al. describe MVTec AD’s objects as always well centred and aligned (Section IV-A); the dataset’s authors write that some objects were placed with random rotations, such as the metal nut, screw and hazelnut (Bergmann et al., 2021, Section 3).

PatchCore. Roth et al. (2022) return to a memory bank but change what goes into it and how it is searched. Each patch feature comes from mid-level blocks (levels 2 and 3 of a Wide ResNet-50, the deeper one bilinearly upsampled) and is averaged over a 3 × 3 neighbourhood of feature positions, which enlarges its receptive field without lowering resolution; the authors argue that the deepest features are too biased towards ImageNet classes and the shallowest too local (Sections 3.1 and 4.4.1, Fig. 4). The features of all normal training images form the memory bank $\mathcal{M}$, which is then reduced by greedy coreset selection: the subset $\mathcal{M}_C$ is chosen to minimize the largest distance from any element of $\mathcal{M}$ to its nearest element of $\mathcal{M}_C$,

$$\mathcal{M}_C^* = \arg\min_{\mathcal{M}_C \subset \mathcal{M}} \; \max_{m \in \mathcal{M}} \; \min_{n \in \mathcal{M}_C} \lVert m - n \rVert_2,$$

a minimax facility-location problem that is NP-hard and is approximated greedily on random low-dimensional projections of the features (Section 3.2). A test patch scores by the distance to its nearest neighbour in the bank, and the image scores by the largest such distance, multiplied by a weight that is larger when the matched bank feature is itself far from its own neighbours, a rare normal pattern (Eq. 7; the text calls this increasing the score, although the printed weight never exceeds 1). Because the bank is shared by all positions, a patch can match good patches from anywhere, which makes the method less dependent on alignment than PaDiM and lets training and test images differ in size (Section 2).

On MVTec AD with 256-pixel resizing, a 224-pixel centre crop and no augmentation, PatchCore with a quarter of the memory bank reached an image AUROC of 99.1%, a pixel AUROC of 98.1% and a PRO score of 93.4%, and kept 99.0%, 98.0% and 93.1% with 1% of the bank (Tables 1–3). The coreset matters more than its size suggests: in the authors’ ablation, the full bank used fewer than 30% of its entries as nearest neighbours at test time, the 1% coreset nearly 95%, and coreset selection beat random subsampling and a set of learned proxies, with a 1% coreset performing comparably to the full bank, while reducing the bank by a larger stride instead lowered image AUROC to 97.6% at stride 2 and 96.8% at stride 3 (Section 4.4.2, Fig. 5). With one-fifth of the normal training images, PatchCore matched the earlier state of the art (Section 4.5). The authors name the method’s limit plainly: it depends on how well the pretrained features transfer to the inspected domain (Section 5). Industrial Visual Inspection gives PatchCore’s inference times.

Normalizing flows

A normalizing flow maps features through an invertible network to a simple base distribution, so it gives an exact likelihood by the change-of-variables formula (see generative models). In anomaly detection it is fitted to pretrained features, not to pixels.

• DifferNet (Rudolph et al., 2021) extracts features from the convolutional part of an ImageNet-pretrained AlexNet at three input scales (448, 224 and 112 pixels), averages each feature map globally into a 768-dimensional vector, and maps it through eight Real NVP coupling blocks to a standard normal; the image score is the negative log-likelihood averaged over several transformations of the image, random rotations and, for one dataset, brightness and contrast changes (Sections 3–4). Localization comes from back-propagating the negative log-likelihood to the input pixels and smoothing the gradient magnitudes (Section 3.3). It reported an image AUROC of 94.9% averaged over MVTec AD, and 87.3% when trained on 16 images per category (Table 1).
• CFLOW-AD (Gudovskiy et al., 2022) keeps the spatial grid: features from three levels of a pretrained encoder are modelled position by position by conditional flows, whose condition is a sinusoidal 2D positional encoding of the position, so one small network per level serves all positions (Section 4). The authors show that under PaDiM’s assumption of Gaussian features, the flow’s training loss can be written in terms of the Mahalanobis distance, and argue that the flow generalizes the Gaussian model to other feature distributions (Section 3.3). With a Wide ResNet-50 encoder they reported 98.26% image AUROC, 98.62% pixel AUROC and 94.60% PRO on MVTec AD, averaged over four runs, with the input resolution (128, 256 or 512 pixels) chosen per category from their ablation, and inference at 27 frames per second at 256 pixels on a GPU (Tables 2, 4 and 6).

Student–teacher methods

Bergmann et al. (2020) turned the comparison of features into a regression problem. A teacher network that outputs a 128-dimensional descriptor for every square patch of side $p$ is first trained on ImageNet patches by distilling the 512-dimensional descriptors of a pretrained ResNet-18, with a term that decorrelates the descriptor’s dimensions; an ensemble of $M = 3$ student networks with the same architecture is then trained, on normal images only, to predict the teacher’s descriptors. At test time a pixel is scored by the students’ regression error against the teacher plus their predictive variance, each normalized on normal validation images; students should agree with the teacher only on the kind of data they were trained on (Section 3). Training teachers and students for $p = 17$, 33 and 65 and averaging the three maps raised the mean PRO area on MVTec AD to 0.914, against 0.857 with $p = 65$ alone (Tables 1 and 3). PatchCore’s and PaDiM’s comparison tables list the student–teacher method at 85.7% PRO (Roth et al., 2022, Table 3; Defard et al., 2021, Table III), which matches the single-scale figure, not the multi-scale one. With updated implementations, the method had the best mean on every threshold-free metric in the 2021 MVTec AD benchmark (Bergmann et al., 2021, Table 3).

Deep one-class objectives and self-supervision

Other methods train a network for the anomaly task directly. Deep SVDD (Ruff et al., 2018) trains a network to map normal images into a hypersphere of minimum volume and scores a test image by its distance from the centre; the network must have no bias terms and the centre must not be learned freely, otherwise it can map every input to the centre, a “hypersphere collapse” (Section 3.3). Patch SVDD, which appears in the comparison tables of Defard et al. (2021) and Gudovskiy et al. (2022), and CutPaste, which appears in Gudovskiy et al.’s, introduce self-supervised training schemes for anomaly detection, as Gudovskiy et al. (Section 2) describe them. Zou et al. (2022) instead change the pretraining: their SPot-the-Difference regularization of contrastive self-supervised ImageNet pretraining treats a copy of an image with small local perturbations as a negative, to make the features sensitive to local changes (Section 3). Adding it raised anomaly segmentation AU-PR on VisA in the two-class high-shot setting by 5.9 points on top of SimSiam and 6.8 on top of supervised pretraining (Table 3), with smaller or mixed changes elsewhere (Tables 2–4).

Example: patch nearest neighbours on brick and grass

The script below implements the memory-bank idea of SPADE and PatchCore with no network, to show the mechanism and how much it depends on the features. Two textures bundled with scikit-image serve as the “product”: brick, a regular wall seen in perspective, and grass, an irregular, stochastic texture. Normal training images are 40 random 64 × 64 crops from the top half of each image; test images are 100 crops from the bottom half, so no test pixel was seen in training. Half the test crops receive one synthetic defect with an exact mask: a scratch (an anti-aliased line, 15–35 pixels long), a spot (a disc of radius 2–5 pixels), both with an intensity change of 0.10–0.25 of random sign, or a missing patch (a square of side 8–16 pixels replaced by its own mean, removing the texture). Two patch features are compared:

1. raw patch: the 49 grey values of the 7 × 7 neighbourhood of each pixel;
2. filter bank: at three scales, the smoothed intensity and the absolute $x$-derivative, $y$-derivative and Laplacian of Gaussian, each averaged locally by a Gaussian of $\sigma = 2$ (a local energy, playing the role of PatchCore’s neighbourhood averaging), 12 channels standardized with the training statistics.

The memory bank holds the features at every second row and column of every training crop (40,960 vectors, no subsampling). Each test feature scores by its Euclidean distance to the nearest bank vector; the map is brought back to pixel resolution and smoothed with $\sigma = 1$ (PatchCore’s $\sigma = 4$ at 224 pixels, scaled to 64), and the image score is its maximum. Pixel AUROC is printed over all test images and over defective images only, because the sources differ: the VisA metric code includes normal images in the localization metrics, which its README notes is different from some other works. Seeds 0, 1 and 2 change the crops and defects.

import numpy as np
from scipy import ndimage as ndi
from skimage import data
from skimage.draw import disk, line_aa
from sklearn.metrics import roc_auc_score

C, STRIDE, N_TRAIN, N_TEST = 64, 2, 40, 100   # crop side, feature stride, crops per set

def crops(tex, rng, n, rows):
    """n random C x C crops whose rows lie in [rows[0], rows[1])."""
    r = rng.integers(rows[0], rows[1] - C + 1, n)
    c = rng.integers(0, tex.shape[1] - C + 1, n)
    return np.stack([tex[i:i + C, j:j + C] for i, j in zip(r, c)])

def add_defect(img, kind, rng):
    """Scratch, spot or missing patch; returns the defective image and its exact mask."""
    img, mask = img.copy(), np.zeros(img.shape, bool)
    amp = rng.uniform(0.10, 0.25) * rng.choice([-1, 1])
    if kind == "scratch":
        r0, c0 = rng.integers(12, C - 12, 2)
        ang, ln = rng.uniform(0, np.pi), rng.integers(15, 36)
        r1 = int(np.clip(r0 + ln * np.sin(ang), 4, C - 5))
        c1 = int(np.clip(c0 + ln * np.cos(ang), 4, C - 5))
        rr, cc, val = line_aa(r0, c0, r1, c1)
        img[rr, cc] += amp * val
        mask[rr[val > 0.3], cc[val > 0.3]] = True
    elif kind == "spot":
        rr, cc = disk(tuple(rng.integers(8, C - 8, 2)), rng.uniform(2, 5), shape=img.shape)
        img[rr, cc] += amp
        mask[rr, cc] = True
    else:                                           # missing patch: texture replaced by its mean
        L = rng.integers(8, 17)
        r, c = rng.integers(6, C - 6 - L, 2)
        img[r:r + L, c:c + L] = img[r:r + L, c:c + L].mean()
        mask[r:r + L, c:c + L] = True
    return img, mask

def raw_features(img):
    """7 x 7 grey-value patch around every STRIDE-th pixel (reflect padding): (h, w, 49)."""
    win = np.lib.stride_tricks.sliding_window_view(np.pad(img, 3, mode="reflect"), (7, 7))
    return win[::STRIDE, ::STRIDE].reshape(C // STRIDE, C // STRIDE, 49)

def bank_features(img):
    """Filter bank: smoothed intensity, |d/dx|, |d/dy|, |LoG| at three scales, locally averaged."""
    ch = []
    for s in (1, 2, 4):
        ch += [ndi.gaussian_filter(img, s), np.abs(ndi.gaussian_filter(img, s, order=(0, 1))),
               np.abs(ndi.gaussian_filter(img, s, order=(1, 0))), np.abs(ndi.gaussian_laplace(img, s))]
    f = np.stack([ndi.gaussian_filter(x, 2) for x in ch], -1)      # local energy over ~a 5 px radius
    return f[::STRIDE, ::STRIDE]

def nn_distance(Q, B, chunk=512):
    """Euclidean distance from each row of Q to its nearest row of B (brute force)."""
    bb, out = (B ** 2).sum(1), []
    for i in range(0, len(Q), chunk):
        q = Q[i:i + chunk]
        d2 = (q ** 2).sum(1)[:, None] - 2 * q @ B.T + bb[None]
        out.append(np.sqrt(np.maximum(d2.min(1), 0)))
    return np.concatenate(out)

def anomaly_maps(feat, train, test):
    Ftr = np.stack([feat(im) for im in train])                       # (N, h, w, d)
    Fte = np.stack([feat(im) for im in test])
    mu, sd = Ftr.reshape(-1, Ftr.shape[-1]).mean(0), Ftr.reshape(-1, Ftr.shape[-1]).std(0)
    if feat is bank_features:                                        # standardize filter channels
        Ftr, Fte = (Ftr - mu) / sd, (Fte - mu) / sd
    d = nn_distance(Fte.reshape(-1, Fte.shape[-1]), Ftr.reshape(-1, Ftr.shape[-1]))
    m = d.reshape(len(test), C // STRIDE, C // STRIDE)
    m = m.repeat(STRIDE, 1).repeat(STRIDE, 2)                        # back to pixel resolution
    return np.stack([ndi.gaussian_filter(x, 1) for x in m])

KINDS = ("scratch", "spot", "missing")
rows, share = [], []
for name in ("brick", "grass"):
    tex = getattr(data, name)().astype(float) / 255
    res = {f: [] for f in ("raw", "bank")}
    for seed in (0, 1, 2):
        rng = np.random.default_rng(seed)
        train = crops(tex, rng, N_TRAIN, (0, 256))
        good = crops(tex, rng, N_TEST // 2, (256, 512))
        bad = [add_defect(im, KINDS[i % 3], rng) for i, im in
               enumerate(crops(tex, rng, N_TEST // 2, (256, 512)))]
        test = np.concatenate([good, np.stack([b[0] for b in bad])])
        masks = np.concatenate([np.zeros_like(good, bool), np.stack([b[1] for b in bad])])
        y = np.r_[np.zeros(len(good)), np.ones(len(bad))]
        kind = np.array([""] * len(good) + [KINDS[i % 3] for i in range(len(bad))])
        sel = {k: (kind == "") | (kind == k) for k in KINDS}       # good images + one defect type
        share.append([masks[kind == k].sum() / masks.sum() for k in KINDS])
        for fname, feat in (("raw", raw_features), ("bank", bank_features)):
            maps = anomaly_maps(feat, train, test)
            s = maps.reshape(len(test), -1).max(1)                    # image score = max of the map
            res[fname].append([roc_auc_score(y, s), *[roc_auc_score(y[sel[k]], s[sel[k]]) for k in KINDS],
                               roc_auc_score(masks.ravel(), maps.ravel()),
                               *[roc_auc_score(masks[sel[k]].ravel(), maps[sel[k]].ravel()) for k in KINDS],
                               roc_auc_score(masks[len(good):].ravel(), maps[len(good):].ravel())])
    rows += [(name, f, np.array(res[f])) for f in ("raw", "bank")]

rng_ = lambda r, j: f"{r[:, j].mean():.3f} ({r[:, j].min():.3f}-{r[:, j].max():.3f})"
print("Image level: AUROC of the max of the map; mean (min-max) over seeds 0-2, per type mean")
print("texture feature | all defects          | scratch  spot   missing")
for name, f, r in rows:
    print(f"{name:7s} {f:7s} | {rng_(r, 0)} | {r[:, 1].mean():.3f}   {r[:, 2].mean():.3f}  {r[:, 3].mean():.3f}")
print("Pixel level: AUROC over pixels; share of defect pixels:",
      ", ".join(f"{k} {v:.2f}" for k, v in zip(KINDS, np.mean(share, 0))))
print("texture feature | all test images      | scratch  spot   missing | defective images only")
for name, f, r in rows:
    print(f"{name:7s} {f:7s} | {rng_(r, 4)} | {r[:, 5].mean():.3f}   {r[:, 6].mean():.3f}  {r[:, 7].mean():.3f}"
          f"   | {rng_(r, 8)}")

Output (NumPy 2.3.5, SciPy 1.16.3, scikit-image 0.25.2, scikit-learn 1.7.2; 3.5 to 10 minutes on our machine, depending on its load; repeated runs gave identical output):

Image level: AUROC of the max of the map; mean (min-max) over seeds 0-2, per type mean
texture feature | all defects          | scratch  spot   missing
brick   raw     | 0.755 (0.730-0.788) | 0.640   0.936  0.685
brick   bank    | 0.666 (0.606-0.708) | 0.564   0.860  0.569
grass   raw     | 0.496 (0.436-0.553) | 0.471   0.535  0.483
grass   bank    | 0.547 (0.522-0.582) | 0.578   0.547  0.515
Pixel level: AUROC over pixels; share of defect pixels: scratch 0.14, spot 0.21, missing 0.65
texture feature | all test images      | scratch  spot   missing | defective images only
brick   raw     | 0.695 (0.661-0.727) | 0.964   0.996  0.541   | 0.691 (0.658-0.722)
brick   bank    | 0.731 (0.708-0.753) | 0.887   0.983  0.620   | 0.724 (0.704-0.746)
grass   raw     | 0.330 (0.266-0.362) | 0.631   0.730  0.133   | 0.333 (0.277-0.364)
grass   bank    | 0.535 (0.484-0.575) | 0.525   0.721  0.485   | 0.528 (0.488-0.567)

• The texture matters more than the feature. On brick, raw patches separate images with spots well at image level (0.936) and the other defects only partly; on grass both features stay close to chance (0.44–0.58 image AUROC over the three seeds). A likely reason, which the script does not test: brick repeats, so 40 training crops contain close matches for almost every normal test patch, whereas no 40 crops of grass cover the variety of its patches, so normal patches lie as far from the bank as defective ones. In our reading, supplying features under which normal patches of natural images do lie close together is the job of the pretrained networks in SPADE, PaDiM and PatchCore. The plan’s expectation that the filter bank would “beat raw patches at image level” on grass held only by 0.05 on average, with overlapping seed ranges.
• The maximum of the map is a harsh image score. On brick with raw patches, scratch and spot pixels rank above most normal pixels (pixel AUROC 0.964 and 0.996), yet the image AUROC for scratches is only 0.640. The image score is the largest value among 4,096 pixels, so a good image with one unusual normal patch can outscore an image with a faint scratch. The plan expected “pixel AUROC is at least as high as image AUROC for each setting”; for the pooled pixel AUROC this held only for brick with the filter bank, while the per-type pixel AUROCs of scratches and spots exceed the corresponding image AUROCs in every row except scratches on grass with the filter bank (0.525 against 0.578).
• Pooled pixel AUROC is dominated by large defects. Missing squares make up 65% of the defect pixels, so the pooled pixel AUROC follows them: on brick with raw patches it is 0.695, between the 0.964–0.996 for scratches and spots and the 0.541 for missing squares. This is Bergmann et al.’s (2021, Section 4.1) objection to pixel-level metrics, and the reason they use PRO. The two features also rank differently by metric: the plan expected raw patches on brick to do “about as well as or better than filter-bank features”, which held at image level, but the filter bank wins on pooled pixel AUROC because it does better on the missing squares that dominate it (0.620 against 0.541).
• A defect that resembles normal texture is missed. A missing square is flat, and the inside of a brick is nearly flat, so on brick both features score missing squares only weakly at pixel level (0.541 and 0.620). On grass, raw patches score them below chance (0.133), presumably because the flat squares lie closer to the bank than ordinary grass does. The plan expected missing squares on grass to be “detected well by the filter bank (low local energy)”; they were not (0.485 pixel, 0.515 image AUROC). The transistor’s missing leg under Challenges is, in our reading, a blind spot of the same kind: an absence that looks normal patch by patch.
• Including the normal images changes little here. The plan expected pixel AUROC “over defective images only is lower than over all test images (normal images add many easy negatives)”; it was lower in three rows and higher for grass with raw patches, and in every row the two differ only in the third decimal, presumably because the normal pixels of defective images resemble those of good images. The VisA README nonetheless flags the choice as a difference between works, so a reported pixel metric should say which it uses.

The first run printed only the pooled pixel AUROCs. A diagnostic run on seed 0 then broke them down by defect type, and the per-type pixel AUROCs and the defect-pixel shares were added to the script, with the table split in two to fit them, so that its own output shows the effect; the data, features, scoring and all other printed numbers are unchanged, and the first run’s values reappear above.

Datasets and Benchmarks

MVTec AD

MVTec AD (Bergmann et al., 2019; extended in Bergmann et al., 2021) is the benchmark on which most methods above report. It has 5,354 images in 15 categories, five textures and ten objects, at side lengths between 700 and 1,024 pixels; three categories (grid, screw and zipper) are grey-scale only. The training set holds 3,629 defect-free images; the test set holds 467 defect-free and 1,258 defective images, with 73 defect types and 1,888 annotated defect regions (Table 1). Each image shows a different physical sample, the defects were made by hand to resemble real ones, and where a part is missing its expected location is annotated as anomalous (Section 3). The protocol is one-class per category: train on that category’s defect-free images, score its test images. The 2021 benchmark split off 10% of the training images of each category as a validation set, the same for all methods (Section 6.1), while later methods report threshold-free metrics on the test set, typically as class averages (Roth et al., 2022, Section 4.1). The official page provides Python evaluation scripts for a consistent comparison and, as of November 2026, releases the data under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 licence, stating that the dataset may not be used for commercial purposes.

Reported results on MVTec AD are now close to the ceiling of the image-level metric: 99.1% for PatchCore and 98.26% for CFLOW-AD in the papers above, with an ensemble of PatchCore models at higher resolution reaching 99.6% (Roth et al., 2022, Table 4). Comparisons across papers must check the protocol: input resolution and cropping, the backbone (PaDiM’s 95.3% and 97.9% above differ only in it), per-category settings such as CFLOW-AD’s resolution, and whether a metric’s area is integrated over all false-positive rates or only up to 0.3.

VisA

VisA (Zou et al., 2022) has 10,821 images, 9,621 normal and 1,200 anomalous, of 12 objects in three groups: four printed circuit boards with complex structure; four subsets with several instances per image (capsules, candles and two kinds of macaroni), of which capsules and one macaroni set vary widely in position and pose; and four roughly aligned single objects (cashew, chewing gum, fryum and pipe fryum). Defects include surface flaws such as scratches, dents, colour spots and cracks and structural ones such as misplaced or missing parts (README). The one-class protocol follows MVTec AD’s: 90% of the normal images of each object for training, and the remaining 10% with all anomalous images for testing. Two-class protocols assign 60% of normal and anomalous images to training (high-shot), or sample 5 or 10 images of each class from a 20% training split, averaged over five random draws (low-shot) (Section 4.2). The authors report the area under the precision–recall curve beside AUROC, arguing that AUROC inflates performance when anomalies, and especially anomalous pixels, are rare. Their PatchCore baseline with a Wide ResNet-50 illustrates this: on VisA its pixel AUROC was 98.4%, higher than its 97.6% on MVTec AD, while its pixel AU-PR was 38.4% against 48.8% (Table 6). As of November 2026, the repository README releases the data under CC BY 4.0.

MVTec LOCO AD and other datasets

MVTec LOCO AD (logical constraints anomaly detection) contains 3,644 images from five categories inspired by industrial inspection, with pixel-precise annotations of both structural and logical anomalies; the official page gives an allowed object in an invalid location and a required object that is absent as examples of logical anomalies, and, as of November 2026, releases the data under CC BY-NC-SA 4.0 with evaluation code. Its paper, Bergmann et al.’s “Beyond Dents and Scratches” (International Journal of Computer Vision, 2022), could not be read here, so its splits and metric are not described. Smaller benchmarks used by the methods above include the Magnetic Tile Defects dataset, on which DifferNet and PatchCore report image AUROC (Rudolph et al., 2021; Roth et al., 2022, Table 6), and frames of the ShanghaiTech Campus videos for localization.

Evaluation Metrics

Image level. The image score is evaluated by the area under the ROC curve (image AUROC), computed per category and averaged on MVTec AD (Roth et al., 2022, Section 4.1). It is threshold-free and insensitive to the share of anomalous images in the test set (Rudolph et al., 2021, Section 4.3), but it says nothing about the error rates at the threshold a line would use; Industrial Visual Inspection discusses those.

Pixel level. Pixel AUROC treats every test pixel as a sample: the true-positive rate is the fraction of anomalous pixels above the threshold and the false-positive rate the fraction of normal pixels above it, summed over the test set (Bergmann et al., 2021, Section 4.1). Because pixels count equally, one large defect found can make up for many small defects missed. The per-region overlap (PRO) instead averages the detected fraction over each connected component $C_{i,k}$ of the ground truth,

$$\mathrm{PRO}(t) = \frac{1}{N} \sum_{i} \sum_{k} \frac{|P_i(t) \cap C_{i,k}|}{|C_{i,k}|},$$

where $P_i(t)$ is the set of pixels of image $i$ above threshold $t$ and $N$ the number of regions (Section 4.2). False positives cannot be assigned to a region, so PRO is plotted against the pixel false-positive rate and the area under that curve is taken up to a limit, 0.3 in the MVTec AD benchmark, and divided by the limit so that the maximum is 1 (AU-PRO; Section 4.3). The authors justify the limit by the imbalance: at a false-positive rate of 30% a segmentation is already degenerate when only 2.7% of MVTec AD’s test pixels are anomalous. The limit changes rankings: in their Table 3, the CNN feature dictionary ranks second by the full area and fourth by the area up to 0.01.

Area under the precision–recall curve. Bergmann et al. (2021, Table 2) also report AU-PR, which does not use the false-positive rate and is integrated over its whole range; precision and recall explain why rare positives keep precision low. Zou et al. (2022, Section 4.2) recommend it for the same reason.

Comparing numbers. Conventions differ in ways that change the values. In Bergmann et al. (2021, Table 3) the ROC area is also integrated only up to a false-positive rate of 0.3 and normalized, whereas PaDiM states a limit only for PRO (Defard et al., 2021, Section IV-A) and the later papers cited here state none for pixel AUROC. Zou et al. (2022, Section 4.2) quote the student–teacher method’s 92.2% AU-ROC from the 2021 benchmark as an example of AUROC overstating performance; by that table’s caption, the figure is the normalized area up to a false-positive rate of 0.3, which, in our reading, is a different quantity from a full-range pixel AUROC. Whether normal test images enter the localization metrics also differs (VisA README).

Applications

• Manufacturing inspection of surfaces, assemblies, electronics and food, the use for which MVTec AD and VisA were built; see Industrial Visual Inspection for the imaging, deployment and threshold questions.
• Medical images. Bergmann et al. (2021, Section 2.2.2) cite unsupervised segmentation of anomalies in brain MR scans with variational autoencoders as one use of the reconstruction approach.
• Active learning. Regions a model finds anomalous can indicate images worth adding to a training set (Bergmann et al., 2021, Section 1; Bergmann et al., 2020, Section 1).
• Unusual events in video, as in the ShanghaiTech Campus frames listed under Variants.

Open Problems

• Logical anomalies and global context. Local patch models do not capture global context (Bergmann et al., 2021, Section 6.2, on the CNN feature dictionary and the student–teacher method), and logical anomalies, which MVTec LOCO AD adds, depend on it.
• Thresholds from normal data. Estimators of a threshold from normal images transfer poorly to test data, and the same estimator gave very different results across methods and categories in the MVTec AD study (Bergmann et al., 2021, Section 6.3); Industrial Visual Inspection discusses the operating thresholds.
• Domain shift of pretrained features. The strongest methods use ImageNet features unchanged, and their reach is limited by how well those features transfer; Roth et al. (2022, Section 5) leave adapting the features to future work, and Zou et al. (2022) show that the choice of pretraining changes the results.
• Saturated benchmarks. With image AUROC above 99% on MVTec AD, differences between methods are fractions of a point. Zou et al. (2022, Section 1) introduced VisA because they judged MVTec AD to be saturating and limited to the one-class setup, and they argue that AU-PR still leaves room where AUROC does not.
• Realism. Benchmark defects are made by hand and photographed in a laboratory (Bergmann et al., 2021, Section 3), so benchmark scores show how methods rank, not the error rates on a particular production line.