Industrial Visual Inspection

Industrial visual inspection uses cameras and vision algorithms to decide whether a manufactured part is acceptable: whether every component is present, whether its surface carries scratches, dents or stains, whether its dimensions are within tolerance, and whether its printed codes are legible and correct. Visual inspection is among the most common non-destructive testing methods, and done by people it is subjective and hard to measure (Czimmermann et al., 2020); an automated system applies the same rule to every part at line speed. The problem differs from photograph classification in three ways that shape every method below: the imaging can be controlled, defects are rare and varied, and the decision is judged by two costs, defective parts that pass and good parts that are thrown away.

Problem

An inspection station sees each part, or each stretch of a continuous web such as sheet metal, film or fabric, and outputs a decision: accept, reject, or send to a human for review. Some stations also report where the defect is and what kind it is, so that a process fault can be traced, or a measurement that feeds statistical process control.

Two errors are possible. This article calls a defective part that is accepted an escape: it reaches the next process or the customer. A good part that is rejected, a false reject, costs material and often a manual re-inspection. The benchmark papers speak of false negatives and false positives; Wang and Qiu (2022, abstract), writing on automated optical inspection of electrical connectors, name “overkill and leakage” as its common problems, which push manufacturers into intensive manual re-inspection. The relative cost of the two errors, not a single accuracy figure, sets where a system operates.

Published surveys and benchmarks cover a range of sectors. Czimmermann et al. (2020) review defect detection on metal, ceramic and textile surfaces. Two research benchmarks draw on others: MVTec AD includes textures such as carpet, leather and wood and objects such as bottles, cables, transistors, pills and capsules (Bergmann et al., 2021), and VisA includes printed circuit boards and food items such as cashews, chewing gum, macaroni and fryums (Zou et al., 2022). The common thread is a product made many times to the same specification, so that “normal” is well defined and anything else is suspect.

Visual Data

Unlike most vision applications, inspection designs its images. The part’s position, the camera, the lens and above all the lighting are chosen so that the feature to be judged has the highest contrast and everything else the lowest.

Lighting

Lighting geometries are named by where the light comes from and whether the directly reflected or transmitted light reaches the lens. The tutorials of two lighting makers, Opto Engineering and Advanced Illumination (both as of October 2026), describe the main ones in the same terms:

• Bright-field front light. Light reflected by a flat surface enters the lens, so the surface looks bright, and scratches or bumps that scatter light outside the lens’s acceptance angle look dark on a bright background.
• Dark-field front light. Light arrives at a low angle, so the directly reflected light misses the lens and only light scattered by non-flat features is collected: scratches, edges and embossing appear bright on a dark background. Low-angle ring lights produce it.
• Back light. The part sits between the light and the camera. An opaque part appears as a dark silhouette, which gives the highest edge contrast for measuring diameters and lengths or locating holes; in a transparent part, features such as air bubbles show up dark. Dark-field back light collects only light scattered by features in a transparent part. Linear back lights serve line-scan cameras on moving film or fabric (Advanced Illumination).
• Coaxial light. Light reaches the surface perpendicularly, along the optical axis, through a beam splitter in front of the lens or a telecentric lens with built-in illumination: flat reflective surfaces look bright and engravings or defects that deflect the light look dark. The more collimated the light, the more contrast it gives on highly reflective surfaces (Opto Engineering).
• Diffuse dome and tunnel lights. Light from almost every direction suppresses reflections on curved, shiny parts; a dome must sit close to the part to keep this effect (Advanced Illumination).

Opto Engineering states the first choice plainly: inspecting a surface or reading printed text needs front light from the camera side, while measuring outlines and holes needs back light. Structured light, a known pattern or a laser line projected onto the part and viewed from another angle, measures the surface’s height rather than its appearance. GenICam’s feature naming convention lists fixed-pattern triangulation cameras among area-scan 3D cameras, and laser-plane (sheet-of-light) triangulation among line-scan 3D devices, which acquire one height profile per trigger while the part moves past them (SFNC 2.8, Chapter 21).

Optics

An ordinary (entocentric) lens has a magnification that changes with the object’s distance, so a part that sits slightly higher looks larger. A telecentric lens accepts only rays parallel to the optical axis, so its magnification does not depend on the object’s position within its depth of field and perspective errors disappear; a lens that is telecentric in both object and image space (bi-telecentric) also keeps the magnification constant across the field depth (Opto Engineering, Telecentric lenses). The price is size: the front element must be at least as large as the field of view, which rules out large parts. The maker’s illumination tutorial recommends collimated back light with telecentric lenses for measuring parts with curved edges, such as shafts, screws and springs, because a diffuse back light reflects off the sides of shiny round parts and makes them look smaller than they are. MVTec AD itself was recorded through two bilateral telecentric lenses with magnifications of 1:5 and 1:1 (Bergmann et al., 2021, Section 3).

Cameras

Inspection uses machine-vision cameras whose exposure, gain and processing can be fixed and whose frames are triggered by the line; the digital camera article describes their interfaces and triggering, and the image sensor and exposure articles the noise and motion-blur trade-offs of short exposures. Two choices are specific to inspection:

• Area-scan or line-scan. An area-scan camera captures a 2D frame; a line-scan camera has a single row of pixels (Opto Engineering, Camera basics) and builds a 2D image as the part or web moves past it. GenICam’s naming convention gives line-scan cameras a line rate (SFNC 2.8, Section 5.5.12) and an encoder interface that triggers lines from a rotary or linear encoder on the conveyor, so that lines are taken at equal steps of distance rather than time (Section 11.1 and Chapter 21). Line-scan suits continuous webs and cylindrical parts rotated in front of the camera.
• Resolution against field of view. The smallest defect to be found sets the pixel size on the part, and the part size then sets the number of pixels. As a rule of reasoning rather than a cited standard, a defect should cover at least a few pixels, which is why large parts are often imaged in several views or with line-scan cameras.

Vision Tasks Involved

• Presence and assembly verification. Is every component there, in the right place and orientation? Missing or misplaced parts are among the structural defects in VisA (Zou et al., 2022), and MVTec AD annotates the expected location of a missing part as anomalous (Bergmann et al., 2021).
• Defect detection and classification. Finding surface defects and, where needed, naming their type. With labelled defects this is image classification, object detection or semantic segmentation on parts.
• Anomaly detection. Flagging anything that differs from good parts, learned from good parts only. Bergmann et al. (2021) motivate it by the two facts of production: it is often unknown in advance which defects will occur, and optimized processes produce few defective samples to learn from.
• Metrology and gauging. Measuring dimensions, hole positions, gaps and angles against tolerances, usually from sub-pixel edge positions (see edge detection) on a calibrated image. Camera calibration and lens distortion correction turn pixels into millimetres; telecentric optics remove the dependence on the part’s distance.
• Reading text and codes. Checking printed lot numbers, dates and labels with optical character recognition, and decoding 1D and 2D codes.
• Alignment. Locating the part before any of the above, by template matching or feature-based image registration, so that regions of interest and reference images line up.

Common Methods

Classical methods

• Golden-template comparison. A reference image of a good part, or the mean of several, is aligned with each new image and subtracted; large differences are defects. The variation model in Steger et al.’s machine vision textbook, as Bergmann et al. (2021, Section 2.2.4) describe it, adds a tolerance per pixel: it stores the mean and the standard deviation of each pixel’s grey value over aligned good images and scores a test pixel by its deviation from the mean in units of that standard deviation. It works only when parts can be aligned precisely: in the MVTec AD benchmark it did well on rigid metal nuts and failed entirely on carpet, where no alignment is possible and the high per-pixel variance accepts almost any grey value (Section 6.2).
• Thresholding and blob analysis. A global or local threshold separates candidate defect pixels, connected-component labelling groups them into blobs, and each blob is accepted or rejected by its area, length, contrast or position. Max-area rules of this kind also appear in the threshold-selection methods of Bergmann et al. (2021, Section 5).
• Morphology. Mathematical morphology cleans masks and, through top-hat transforms, extracts small bright or dark spots against an uneven background.
• Texture models. For regular textures such as fabric, statistical, structural, filter-based and model-based texture descriptors flag regions whose texture differs from the normal pattern; Czimmermann et al. (2020, Section 4) survey them. The GMM-based texture inspection model benchmarked on MVTec AD fits a Gaussian mixture to patches of good texture on several pyramid levels and scores pixels by negative log-likelihood (Bergmann et al., 2021, Section 6.1).
• Rules. Measurements and blob features are combined into explicit accept rules taken from the product’s specification.

Classical pipelines are built from simple operations whose parameters an engineer sets; they are fast, explainable and need few images, but each rule is written for one product and one lighting set-up, and they struggle with natural variation (wood grain, food, deformable parts) that no single tolerance describes.

Supervised defect models

When labelled defect images exist, the standard tasks apply: a classifier for accept or reject or for the defect type, a detector for defect boxes, a segmenter such as U-Net for defect masks. Transfer learning from ImageNet-pretrained backbones and data augmentation offset the small number of defect images. Zou et al. (2022) note that in mature manufacturing domains anomalous samples are available and sometimes sufficient, and that a standard two-class model trained on them can then improve on methods that use good parts only; their VisA two-class protocols train on 5 or 10 images of each class (low-shot) or on 60% of the images (high-shot). The weakness of supervised models is the defect they have not seen: a new failure mode of the process is, by construction, outside the training classes.

Anomaly detection from good parts only

Anomaly detection learns what good parts look like and scores how far a test image departs from it, producing an anomaly map whose maximum, or another summary, scores the image. Three families are common among methods benchmarked on MVTec AD.

• Reconstruction. An autoencoder or a generative adversarial network trained on good images should reconstruct defects poorly, so the difference between input and reconstruction marks them. Bergmann et al. (2021, Section 6.2) found that such models trained from scratch often reconstruct fine detail poorly, producing false positives on object boundaries and fine structures such as a toothbrush’s bristles, and can miss a defect whose reconstruction differs little from the input.
• Feature-distribution models on pretrained networks. Patches are described by the activations of a network pretrained on ImageNet, such as a ResNet, and the distribution of good-patch descriptors is modelled. PaDiM (Defard et al., 2021) fits one multivariate Gaussian to the embedding vectors at each patch position, estimated over all training images with a small regularizing term added to the covariance, and scores a test patch by its Mahalanobis distance to that Gaussian; the image score is the maximum of the map. Because every position has its own model, PaDiM assumes roughly aligned images: on a version of MVTec AD with random rotations of up to ±10° and random crops, its pixel-level AUROC fell from 97.5% to 92.2%, a smaller drop than for the SPADE and variational-autoencoder baselines they compared (arXiv version, Tables III and VI). PatchCore (Roth et al., 2022) instead pools locally aggregated mid-level patch features of all good images into one memory bank, reduces it by greedy coreset subsampling, and scores each test patch by the distance to its nearest neighbour in the bank, so that a patch can match good patches from any position. Keeping 25% of the memory bank, the authors reported an image-level AUROC of 99.1% averaged over the MVTec AD classes (against 95.3% for PaDiM with the same WideResNet-50 backbone and 97.9% for PaDiM with a backbone chosen per problem), a pixel-level AUROC of 98.1% and a PRO score of 93.4%, with 42 of the 1,725 test images misclassified at the F1-optimal threshold (Tables 1–3); an ensemble at higher resolution, with a 1% memory bank, reached 99.6% image-level AUROC (Table 4).
• Normalizing flows. A normalizing flow learns an invertible map from features to a simple distribution, so it gives an explicit likelihood. DifferNet (Rudolph et al., 2021) estimates the density of multi-scale features of a pretrained network with a normalizing flow, scores images by their likelihood under several transformations, and localizes defects by propagating the score back to the input.

A student–teacher method, in which student networks trained on good images learn to regress a pretrained teacher’s descriptors and fail on anomalies, had the best mean scores on every threshold-free metric in the benchmark of Bergmann et al. (2021, Table 3), ahead of the feature dictionary, texture model, autoencoders, variation model and f-AnoGAN. In the 2019 conference version, which predates the student–teacher method, the two autoencoders had done best on most object categories (Bergmann et al., 2019, Section 4.4.2). The authors concluded in 2021 that methods using pretrained descriptors outperformed generative models trained from scratch.

Benchmarks

MVTec AD (Bergmann et al., 2019; extended journal version 2021) has 5,354 colour images in 15 categories, five textures and ten objects, with 3,629 good images for training and 1,725 test images, good and defective, covering 73 defect types and 1,888 annotated defect regions (Bergmann et al., 2021, Table 1). Each image shows a different physical sample, the defects were made by hand to resemble real ones, and some categories were placed with random rotations while others are roughly aligned (Section 3). As of October 2026 the official page releases the data under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 licence and states that the dataset may not be used for commercial purposes; the journal paper itself is open access under CC BY 4.0.

VisA (Zou et al., 2022) has 10,821 images, 9,621 good and 1,200 anomalous, of 12 objects in three groups: four kinds of printed circuit board with complex structure, four subsets with several instances per image (capsules, candles and two kinds of macaroni), some with large variation in position and pose, and four roughly aligned single objects. Its one-class protocol follows MVTec AD; it also defines two-class protocols. In the authors’ PaDiM baseline with supervised ImageNet pretraining, the anomaly-segmentation AU-PR averaged 11.4% on VisA against 35.2% on MVTec AD (Table 2); the authors attribute VisA’s difficulty to complex structures, multiple instances and large variation in position and scale (Fig. 6). As of October 2026 the repository README releases the data under CC BY 4.0.

Both datasets contain hand-made defects photographed in a laboratory. Results on them show how methods rank on a common task; they do not predict the escape rate of a station on a particular line, which depends on that line’s parts, lighting and defects (see dataset bias).

Example

The script renders a textured part, a woven pattern with a period of 8 pixels, with small changes of gain and offset between parts and sensor noise. Defective parts carry one or two dark scratches or small bright or dark spots, with a contrast between 0.08 and 0.2, weaker than the texture’s own peak-to-peak swing of about 0.5, and an exact ground-truth mask. Each run renders 150 good training parts, 50 good validation parts, and a test set of 100 good and 100 defective parts. Two inspectors are compared:

1. Golden template: the pixelwise mean of the training parts, with the anomaly map equal to the absolute difference from it, slightly smoothed.
2. Patch Gaussians: simple local features (intensity, squared gradients, absolute Laplacian, local contrast) averaged over 8 × 8 patches at a stride of 4, with one multivariate Gaussian fitted per patch position and the Mahalanobis distance as the patch score. This is PaDiM’s statistical model on hand-made features instead of a pretrained network.

Each method’s image score is the maximum of its map. The script reports image-level AUROC, pixel-level AUROC and the area under the PRO curve up to a false-positive rate of 0.3 (defined under Evaluation); the false-reject rate at zero escapes, the fraction of good test parts rejected by the threshold that just passes no defective part; and the false-reject and escape rates at a threshold set from good parts only, the maximum score over the validation parts. It runs every setting for five seeds, first with perfectly aligned parts and then with each part shifted by up to ±2 pixels, a quarter of a texture period, and rotated by up to ±1°.

import numpy as np
from scipy import ndimage as ndi
from skimage.draw import disk, line_aa
from sklearn.metrics import roc_auc_score

S, P, STRIDE = 128, 8, 4          # image side, patch side (one texture period), patch stride

def render(rng, shift):
    """A textured part: periodic weave (period 8 px), gain and offset jitter, sensor noise."""
    dx, dy = rng.uniform(-shift, shift, 2)
    th = np.radians(rng.uniform(-shift, shift) / 2)                 # small rotation with the shift
    y, x = np.mgrid[0:S, 0:S].astype(float) - S / 2
    u, v = np.cos(th) * x - np.sin(th) * y - dx, np.sin(th) * x + np.cos(th) * y - dy
    tex = 0.2 * np.sin(2 * np.pi * u / 8) * np.sin(2 * np.pi * v / 8) + 0.08 * np.cos(2 * np.pi * (u + v) / 16)
    img = 0.5 + rng.uniform(0.95, 1.05) * tex + rng.uniform(-0.02, 0.02)
    return img + rng.normal(0, 0.02, img.shape)

def add_defects(img, rng):
    """One or two scratches (thin dark lines) or spots (small discs); returns image and mask."""
    mask = np.zeros(img.shape, bool)
    for _ in range(rng.integers(1, 3)):
        a = rng.uniform(0.08, 0.2) * rng.choice([-1, 1])           # contrast, weaker than the texture
        if rng.random() < 0.5:
            r0, c0 = rng.integers(16, S - 16, 2)
            ang, ln = rng.uniform(0, np.pi), rng.integers(12, 30)
            r1, c1 = int(r0 + ln * np.sin(ang)), int(c0 + ln * np.cos(ang))
            rr, cc, val = line_aa(r0, c0, np.clip(r1, 0, S - 1), np.clip(c1, 0, S - 1))
            img[rr, cc] -= abs(a) * val                            # scratches are dark
            mask[rr[val > 0.3], cc[val > 0.3]] = True
        else:
            rr, cc = disk(tuple(rng.integers(10, S - 10, 2)), rng.uniform(1.5, 3.5), shape=img.shape)
            img[rr, cc] += a
            mask[rr, cc] = True
    return img, mask

def dataset(rng, shift, n_train=150, n_val=50, n_test=100):
    train = np.stack([render(rng, shift) for _ in range(n_train)])
    val = np.stack([render(rng, shift) for _ in range(n_val)])
    good = [render(rng, shift) for _ in range(n_test)]
    bad = [add_defects(render(rng, shift), rng) for _ in range(n_test)]
    test = np.stack(good + [b[0] for b in bad])
    masks = np.stack([np.zeros((S, S), bool)] * n_test + [b[1] for b in bad])
    return train, val, test, masks

# Method 1: golden template, a pixelwise mean of good parts; score = smoothed absolute difference.
def template_maps(train, imgs):
    golden = train.mean(0)
    return np.stack([ndi.gaussian_filter(np.abs(im - golden), 1) for im in imgs])

# Method 2: one Gaussian per patch location on simple local features (PaDiM-like, no network).
def features(img):
    gx, gy = ndi.sobel(img, 1), ndi.sobel(img, 0)
    lap = ndi.laplace(img)
    m = ndi.uniform_filter(img, 3)
    f = np.stack([img, gx ** 2, gy ** 2, np.abs(lap), (img - m) ** 2, np.minimum(img - m, 0)])
    # average each feature over P x P patches (one texture period) at the given stride
    pooled = ndi.uniform_filter(f, (1, P, P))[:, P // 2::STRIDE, P // 2::STRIDE]
    return pooled[:, : (S - P) // STRIDE + 1, : (S - P) // STRIDE + 1].transpose(1, 2, 0)

def fit_patch_gaussians(train, eps=1e-6):
    F = np.stack([features(im) for im in train])                     # (N, h, w, d)
    mu = F.mean(0)
    Xc = F - mu
    cov = np.einsum("nhwi,nhwj->hwij", Xc, Xc) / (len(F) - 1)
    cov += eps * np.trace(cov, axis1=2, axis2=3)[..., None, None] * np.eye(F.shape[-1])
    return mu, np.linalg.inv(cov)

def patch_maps(model, imgs):
    mu, icov = model
    out = []
    for im in imgs:
        d = features(im) - mu
        m = np.sqrt(np.einsum("hwi,hwij,hwj->hw", d, icov, d))          # Mahalanobis distance
        full = np.zeros((S, S)); cnt = np.zeros((S, S))
        for i in range(m.shape[0]):                                   # spread each patch score
            for j in range(m.shape[1]):                               # over the pixels it covers
                full[i * STRIDE:i * STRIDE + P, j * STRIDE:j * STRIDE + P] += m[i, j]
                cnt[i * STRIDE:i * STRIDE + P, j * STRIDE:j * STRIDE + P] += 1
        out.append(full / cnt)
    return np.stack(out)

def au_pro(maps, masks, fpr_max=0.3):
    """Normalized area under the per-region-overlap curve up to fpr_max (Bergmann et al., 2021)."""
    neg = np.sort(maps[~masks])
    ts = neg[::-1][np.linspace(0, int(fpr_max * len(neg)) - 1, 300).astype(int)]
    fpr = 1 - np.searchsorted(neg, ts, side="right") / len(neg)
    regions = []
    for mp, mk in zip(maps, masks):
        lab, n = ndi.label(mk, np.ones((3, 3)))
        regions += [np.sort(mp[lab == k]) for k in range(1, n + 1)]
    pro = np.mean([1 - np.searchsorted(r, ts, side="right") / len(r) for r in regions], 0)
    return np.trapezoid(pro, fpr) / fpr_max

def evaluate(maps, val_maps, masks, n_good):
    s = maps.reshape(len(maps), -1).max(1)                           # image score = max of the map
    y = np.r_[np.zeros(n_good), np.ones(len(maps) - n_good)]
    good, bad = s[:n_good], s[n_good:]
    frr_zero_esc = np.mean(good >= bad.min())                        # threshold that passes no defect
    t_val = val_maps.reshape(len(val_maps), -1).max(1).max()         # threshold from good parts only
    return dict(img_auc=roc_auc_score(y, s), pix_auc=roc_auc_score(masks.ravel(), maps.ravel()),
                pro=au_pro(maps, masks), frr0=frr_zero_esc,
                val_frr=np.mean(good > t_val), val_esc=np.mean(bad <= t_val))

print("shift method    | img AUROC pix AUROC AU-PRO | FRR@0 esc | val thr: FRR  escape")
for shift in (0.0, 2.0):
    for seed in range(5):
        rng = np.random.default_rng(seed)
        train, val, test, masks = dataset(rng, shift)
        n_good = len(test) // 2
        model = fit_patch_gaussians(train)
        for name, fn in (("template", lambda x: template_maps(train, x)),
                         ("patch-gauss", lambda x: patch_maps(model, x))):
            r = evaluate(fn(test), fn(val), masks, n_good)
            print(f"{shift:4.1f}  {name:11s} s{seed} | {r['img_auc']:9.3f} {r['pix_auc']:9.3f} {r['pro']:6.3f} |"
                  f" {r['frr0']:7.2f}   |        {r['val_frr']:5.2f} {r['val_esc']:6.2f}")

Output (NumPy 2.3.5, SciPy 1.16.3, scikit-image 0.25.2, scikit-learn 1.7.2; about 20 s; two runs gave identical output):

shift method    | img AUROC pix AUROC AU-PRO | FRR@0 esc | val thr: FRR  escape
 0.0  template    s0 |     0.999     0.997  0.993 |    0.03   |         0.03   0.01
 0.0  patch-gauss s0 |     0.988     0.990  0.977 |    0.62   |         0.00   0.04
 0.0  template    s1 |     0.992     0.998  0.994 |    0.48   |         0.03   0.04
 0.0  patch-gauss s1 |     0.958     0.987  0.959 |    0.97   |         0.02   0.11
 0.0  template    s2 |     0.995     0.999  0.996 |    0.30   |         0.00   0.03
 0.0  patch-gauss s2 |     0.991     0.994  0.981 |    0.46   |         0.00   0.05
 0.0  template    s3 |     0.993     0.998  0.994 |    0.53   |         0.02   0.06
 0.0  patch-gauss s3 |     0.990     0.991  0.971 |    0.63   |         0.00   0.05
 0.0  template    s4 |     0.999     0.999  0.996 |    0.05   |         0.00   0.05
 0.0  patch-gauss s4 |     0.998     0.993  0.980 |    0.24   |         0.01   0.01
 2.0  template    s0 |     0.681     0.707  0.429 |    0.99   |         0.02   0.84
 2.0  patch-gauss s0 |     0.974     0.982  0.957 |    0.85   |         0.05   0.08
 2.0  template    s1 |     0.668     0.719  0.438 |    1.00   |         0.04   0.83
 2.0  patch-gauss s1 |     0.942     0.972  0.918 |    0.80   |         0.03   0.25
 2.0  template    s2 |     0.628     0.716  0.455 |    0.95   |         0.03   0.84
 2.0  patch-gauss s2 |     0.975     0.979  0.944 |    0.77   |         0.00   0.08
 2.0  template    s3 |     0.655     0.708  0.419 |    1.00   |         0.00   0.93
 2.0  patch-gauss s3 |     0.953     0.974  0.930 |    0.96   |         0.03   0.19
 2.0  template    s4 |     0.606     0.716  0.432 |    0.93   |         0.02   0.88
 2.0  patch-gauss s4 |     0.966     0.985  0.959 |    0.84   |         0.03   0.17

• Aligned parts favour the template. With perfect alignment the mean image subtracts the texture exactly, leaving only noise and gain jitter, so every pixel of a defect stands out; the template scores best on every threshold-free metric in all five seeds. The patch model averages each feature over a whole texture period, which dilutes a one-pixel scratch, and its maps are coarser. This matches the MVTec AD benchmark, where the variation model did well on rigid, precisely aligned metal nuts (Bergmann et al., 2021).
• A quarter-period misalignment breaks the template. Shifted by up to two pixels, the texture no longer cancels: the difference from the mean image is as large on good parts as on the defects, the template’s image-level AUROC falls to 0.61–0.68 and its AU-PRO to 0.42–0.46, and a threshold set on good parts lets 83–93% of defective parts through. The patch model changes little (image AUROC 0.94–0.98, AU-PRO 0.92–0.96), because features averaged over one 8-pixel period depend little on where the window starts: for the period-8 part of the weave the window always covers whole periods, and only terms involving the weaker 16-pixel component change with the shift. The mean image of misaligned parts is also blurred, so collecting more training images does not help; the template needs registration to a precision well below the texture period, which is why Bergmann et al. (2021, Section 6.1) aligned six MVTec AD categories before applying the variation model.
• A high AUROC does not mean few rejects at zero escapes. Image-level AUROCs of 0.96–1.00 on aligned parts coexist with false-reject rates at zero escapes of 0.03–0.53 for the template and 0.24–0.97 for the patch model. Zero escapes on this test set requires a threshold at or below the score of the lowest-scoring defective part of the hundred, and that part scores within the range of good parts. AUROC averages over all pairs of good and defective parts; the zero-escape operating point depends on the single worst one, and it varies more from seed to seed than any AUROC in the table.
• Thresholds from good parts only trade the other way. The maximum score over 50 good validation parts rejects 0–3% of good test parts but lets 1–11% of defective parts through on aligned images with either method; with the patch model on misaligned parts it rejects 0–5% and lets 8–25% through. Fifty good parts do not bound the tail of the good-part score distribution precisely, and nothing in them says where the faintest defect will score.

Challenges

• Few defects, many kinds. New failure modes can appear after deployment, so a defect training set is never complete. This is the reason for anomaly detection, and also its limit: a method trained on good parts cannot tell an unseen harmless variation from an unseen defect.
• Imbalance at two levels. Defective parts are rare among parts, and defective pixels are rare within images: only 2.7% of MVTec AD’s test pixels are anomalous, so thresholds with a high false-positive rate give meaningless segmentations (Bergmann et al., 2021, Section 4.3). Precision and recall explain why rare positives make precision fragile.
• Small and subtle defects. Defects range from large structural changes to thin scratches and colour faults; their areas in MVTec AD vary so widely that Bergmann et al. (2021, Fig. 3) plot them on a logarithmic scale, and Zou et al. (2022) cite inspection tasks down to damaged millimetre-sized components on a complex circuit board.
• Natural variation. Wood grain, food, cables and leather vary from part to part in ways that are acceptable, and a model must learn that variation without absorbing the defects.
• Alignment and pose. Parts that arrive in any orientation, or several per image as in VisA, defeat template and per-position models and need methods that compare patches regardless of position.
• Drift. Lighting ages, lenses get dirty, suppliers change materials and the process itself shifts. It follows from the one-class setting (reasoning, not a cited result) that any such change can raise anomaly scores on good parts, which shows as a sudden rise in rejects across the line, while a supervised model’s errors under the same change are harder to predict; both need monitoring after deployment.
• Labels. Whether a mark is a defect is often a judgement about size relative to the part, and Czimmermann et al. (2020, Sections 1 and 2) call the classification of a defect a severely subjective judgement and recommend fixing a quality standard before designing the system.

Evaluation

Image level. Anomaly detection papers summarize image scores by the ROC AUC averaged over categories (Roth et al., 2022, Section 4.1). A line runs at a single threshold, and the numbers that matter there are the escape rate (defective parts accepted, the false-negative rate of the “defective” class) and the false-reject rate (good parts rejected), read from the confusion matrix at that threshold. A specification may fix one, for example an escape rate the customer accepts, and asks for the lowest achievable value of the other; the false-reject rate at zero escapes in the example is the strictest form of that question. As the roc-auc article notes, an AUROC of 0.99 can still let too many defects through at the false-alarm rate a line can tolerate.

Pixel level. Pixel-level AUROC treats every pixel as a sample, with the true-positive rate the fraction of anomalous pixels flagged and the false-positive rate the fraction of normal pixels flagged. Bergmann et al. (2021, Section 4.1) point out that this favours large defects: finding one large defect can make up for missing several small ones. The per-region overlap (PRO) instead averages the detected fraction over each connected component of the ground truth:

$$\mathrm{PRO}(t) = \frac{1}{N} \sum_{i} \sum_{k} \frac{|P_i(t) \cap C_{i,k}|}{|C_{i,k}|},$$

where $C_{i,k}$ is the $k$-th defect region of test image $i$, $P_i(t)$ the pixels predicted anomalous at threshold $t$, and $N$ the number of defect regions. Plotted against the pixel false-positive rate and integrated up to a limit (0.3 in their benchmark), it gives AU-PRO, normalized so that the maximum is 1 (Bergmann et al., 2021, Sections 4.2–4.3). The limit matters: in their Table 3, lowering it from 0.3 to 0.01 cuts the methods’ AU-PRO by factors of about 2 (student–teacher) to 8 (SSIM autoencoder) and changes their ranking, so many methods find defects only while also flagging many good pixels. The text of Section 6.2 gives factors of 2 to 11, which match the table when the full area (limit 1.0) is compared with 0.01.

Thresholds. Bergmann et al. (2021, Sections 5 and 6.3) compared four ways of setting a threshold from good validation images only, on three categories: the maximum score, a quantile, the mean plus a multiple of the standard deviation, and the lowest threshold that leaves no false-positive region above a maximum area. Thresholds aimed at a 1% false-positive rate gave much higher rates on the test set, the maximum was so conservative that it sometimes detected nothing, and the same estimator gave very different results across methods and categories; they concluded that choosing a threshold from good data alone is a highly challenging problem in practice. The 2019 conference version of MVTec AD had already reported image classification accuracy and per-region overlap at such an estimated threshold, next to the threshold-free pixel ROC AUC (Bergmann et al., 2019, Section 4.3).

Test sets. Estimating a low escape rate needs many defective parts: with 100 defective test parts, as in the example, zero observed escapes is still consistent with a true escape rate of about 3%. The one-sided 95% upper bound $p$ for zero failures in $n$ independent trials solves $(1-p)^n = 0.05$, so $p = 1 - 0.05^{1/n} \approx 3/n$ (2.95% for $n = 100$), the binomial “rule of three”. It follows (reasoning, not a cited result) that escape rates claimed at the parts-per-million level require either very large validated sets or a separate argument, such as a physical limit on what the optics can miss; statistical significance in evaluation gives the confidence intervals.

Failure Modes and Risks

• Escapes of unseen defects. A supervised model passes defect types absent from training; an anomaly model can pass a defect that resembles normal texture, or a logical error such as a correct-looking component in the wrong place, since local patch models do not capture global context: the student–teacher method missed a transistor’s missing leg and the feature dictionary a cable whose insulation had the wrong colour (Bergmann et al., 2021, Section 6.2).
• Reflections and boundaries. Strong reflections and object edges raise anomaly scores in many methods (Bergmann et al., 2021, Section 6.2), a reason why lighting design comes first.
• Grey-scale blind spots. Methods that convert to grey scale miss colour defects; the SSIM autoencoder and the GMM texture model missed colour defects on MVTec AD for this reason (Bergmann et al., 2021, Section 6.2).

Deployment Considerations

• Cycle time. Every part must be judged before the next arrives. Bergmann et al. (2021, Section 6.4) report only qualitative runtimes, since they depend on implementation: from a few milliseconds for single-pass networks on a GPU to several seconds per image for a patch-wise CNN feature dictionary, and tens to hundreds of milliseconds for CPU implementations of the variation and texture models. Roth et al. (2022, Table 5) report mean inference times of 0.17 s per image for PatchCore with a 1% memory bank and 0.19 s for their reimplementation of PaDiM on MVTec AD; coreset subsampling exists to make the nearest-neighbour search fit the time budget.
• Integration and fixed imaging. The decision drives a reject mechanism, and results go to the line’s control and quality systems. Exposure, gain, white balance and lighting intensity stay fixed rather than automatic, so that a part looks the same in every frame; HDR imaging gives the options for shiny parts with a wide dynamic range.
• Thresholds and review. Thresholds are set from the costs of the two errors and validated on as many real defective parts as can be collected; a band of uncertain scores can be sent to human review rather than forced into accept or reject. Wang and Qiu (2022) report that, in an electrical connector plant, total inspection cost was lowest when the inspection strictness was set close to the true defect rate.
• Monitoring and retraining. Reject rates, score distributions and samples of accepted parts are tracked over time, so that drift and new defect types are found by the process and not by the customer.