Viola-Jones Object Detection

The Viola–Jones detector is an object detection method that finds objects, classically upright frontal faces, by running a very fast classifier over every window of an image. Paul Viola and Michael Jones built it from three parts: Haar-like rectangle features computed in constant time from an integral image; a version of AdaBoost that builds each classifier from a small number of those features, one feature per weak classifier, so that boosting doubles as feature selection; and an attentional cascade, a chain of such classifiers in which each window must pass every stage and most windows are rejected by the first one or two. Because a typical window is background and the early stages are tiny, the detector spends a few feature evaluations on most windows and its full effort only on the rare windows that look like the object. Its face detector processed $384 \times 288$ images at 15 frames per second on a 700 MHz Pentium III, about 15 times faster than the Rowley–Baluja–Kanade neural network detector by the authors’ measurement, and boosted cascades of this kind remain in OpenCV 4.14.0 (as of October 2026). The method is also called the boosted cascade of simple features, or Haar cascade.

The features and the integral image are covered in Haar-like features, the scan over positions and scales and the merging of detections in sliding window detection, the 2001 publication and its experiments in the paper article, and face detection as a task in face detection.

Problem

A sliding window detector classifies every window at every position and scale; Viola and Jones’s NIPS 2001 paper counts about 50,000 in a typical image, and almost all are background. Two demands follow. The window classifier must be accurate at very low false-positive rates: at 50,000 windows, a rate of $10^{-4}$ per window still means about five false detections per image. And it must be cheap, since its cost is paid for every window. Making a single classifier more accurate usually means giving it more features, which makes every window more expensive. Viola and Jones reported that a single boosted classifier with 200 features reached a 95% detection rate at a false-positive rate of 1 in 14,084 and took 0.7 s per $384 \times 288$ image; in the journal version they note that a practical detector needs a false-positive rate closer to 1 in 1,000,000.

The cascade removes the conflict by spending computation unequally. Most windows are easy to reject, and the few that are hard can afford a large classifier because there are so few of them.

Inputs and Outputs

Training takes positive examples, image windows of the object aligned and scaled to a base resolution ($24 \times 24$ pixels for Viola and Jones’s faces), and a set of images known not to contain the object, from which negative windows are drawn. Each window is normalized to unit variance before its features are computed. Training also takes the per-stage targets described below. It returns a cascade: a list of stages, each a set of weighted single-feature classifiers with a stage threshold.

Detection takes a grayscale image and returns the windows that passed every stage, usually after overlapping detections are merged. Each window gets a yes or no. To trace an ROC curve, Viola and Jones varied the threshold of the last stage and then removed stages one at a time.

Intuition

A single rectangle feature is a weak test. Placed across the eyes and cheeks of a face, it fires when a dark band sits above a lighter one, which is true of almost every face but also of many window frames and shadows. Boosting combines many such tests: it picks the feature that best separates faces from background on the current training weights, raises the weights of the examples that feature gets wrong, picks the next feature for those, and so on, and the final classifier is a weighted vote.

The cascade adds the observation that rejecting a window needs less evidence than accepting it. A two-feature classifier whose threshold is set low enough to accept practically every face still rejects a large share of background windows, about half in the journal paper. A second, slightly larger classifier trained only on what survives the first rejects half of the rest, and so on. A face has to pass every stage, but background usually drops out within the first few, so the average number of features evaluated per window stays close to the size of the first stages, however large the late stages become.

The cascade also changes what the later stages learn. Each stage is trained on the background windows that fooled all earlier stages, so later stages see harder and harder negatives.

Algorithm

Boosting a stage

Viola and Jones’s boosting procedure (Table 1 in both the conference and the journal paper) is the discrete AdaBoost of Freund and Schapire with the weak learner restricted to one feature:

1. Start with $l$ positive and $m$ negative examples, labels $y_i = 1$ and $y_i = 0$, and initial weights $\frac{1}{2l}$ for positives and $\frac{1}{2m}$ for negatives, so that the two classes carry equal total weight.
2. For $t = 1, \dots, T$:
   1. Normalize the weights to sum to 1.
   2. For every feature, find the threshold and parity with the lowest weighted error (below); choose the feature with the lowest error $\epsilon_t$ of all.
   3. Multiply the weight of every correctly classified example by $\beta_t = \epsilon_t / (1 - \epsilon_t)$, leaving the misclassified ones unchanged.
3. The strong classifier accepts a window when the weighted vote of the $T$ weak classifiers, with votes $\alpha_t = \log(1/\beta_t)$, reaches half the total vote.

The best threshold for a feature takes one pass over the examples sorted by that feature’s value. Keeping running sums of the positive and negative weight below the current position, the error of a threshold placed there is the smaller of two quantities: the positive weight below plus the negative weight above (call everything above positive), or the converse. The journal paper describes this search.

Training the cascade

The journal paper gives the cascade training procedure as its Table 2; the conference paper describes the same scheme in a paragraph. The user chooses $f$, the largest acceptable false-positive rate of a stage, $d$, the smallest acceptable detection rate of a stage, and an overall target $F_{\text{target}}$. Then:

1. Start with $F_0 = D_0 = 1$, the positive set, and negatives drawn at random from the face-free images.
2. Add a stage. Train it with AdaBoost one feature at a time; after each feature, lower the stage threshold until the cascade so far detects at least $d \cdot D_{i-1}$ of a validation set, and measure its false-positive rate $F_i$ on validation data. Stop adding features once $F_i \le f \cdot F_{i-1}$.
3. If $F_i$ is still above $F_{\text{target}}$, discard the negatives, run the current cascade over the face-free images, keep the windows it wrongly accepts as the new negatives, and go to step 2.

Step 3 is the source of the cascade’s hard negatives: each stage is trained on the false positives of all the stages before it. Lowering the threshold in step 2 trades false positives for detections. AdaBoost’s own threshold, half the total vote, minimizes training error, which is the wrong goal for an early stage that must almost never reject a face.

Running the cascade

During detection each window is passed to stage 1. A stage computes its features, sums the votes of its weak classifiers and compares the sum with its threshold; a window below the threshold is rejected at once, and a window above it goes to the next stage. Windows that pass the last stage are detections. Feature values are divided by the window’s standard deviation, computed from integral images of the image and its square; the Haar-like features article covers this normalization, and sliding window detection covers how Viola and Jones scaled the detector instead of the image, the step size, and how they merged overlapping detections.

Mathematical Formulation

Single-feature weak classifiers

Let $\phi_j(x)$ be the value of rectangle feature $j$ on window $x$ (Viola and Jones write $f_j$). A weak classifier consists of a feature, a threshold $\theta_j$ and a parity $p_j \in \{-1, +1\}$ that sets the direction of the inequality:

$$h_j(x) = \begin{cases} 1 & \text{if } p_j \phi_j(x) < p_j \theta_j, \\ 0 & \text{otherwise.} \end{cases}$$

The journal paper notes that these thresholded single features are decision stumps, single-node decision trees. With $N$ training examples, only $N$ thresholds per feature are distinct, so a pool of $K$ features defines about $KN$ different weak classifiers; the journal paper’s example, $K = 160{,}000$ and $N = 20{,}000$, gives 3.2 billion.

AdaBoost

With normalized weights $w_{t,i}$, round $t$ chooses the weak classifier with the lowest weighted error

$$\epsilon_t = \min_{j,\, p,\, \theta} \sum_i w_{t,i} \, \lvert h(x_i; j, p, \theta) - y_i \rvert,$$

updates the weights by

$$w_{t+1,i} = w_{t,i} \, \beta_t^{\,1 - e_i}, \qquad \beta_t = \frac{\epsilon_t}{1 - \epsilon_t},$$

with $e_i = 0$ if example $i$ is classified correctly and $e_i = 1$ otherwise, and after $T$ rounds gives the strong classifier

$$C(x) = \begin{cases} 1 & \text{if } \sum_{t=1}^{T} \alpha_t h_t(x) \ge \tau, \\ 0 & \text{otherwise,} \end{cases} \qquad \alpha_t = \log \frac{1}{\beta_t},$$

with $\tau = \frac{1}{2}\sum_t \alpha_t$ in plain AdaBoost and a lower $\tau$ in a cascade stage. Since $\epsilon_t < \frac{1}{2}$, $\beta_t < 1$: correctly classified examples lose weight, and after normalization the mistakes gain it. A weak classifier with lower error gets a larger vote.

The weight update and the final vote are those of AdaBoost as Freund and Schapire defined it. They proved that the error of the combined classifier on the training set, weighted by the initial distribution, is at most

$$\epsilon \le 2^T \prod_{t=1}^{T} \sqrt{\epsilon_t (1 - \epsilon_t)}.$$

If every weak error is at most $\frac{1}{2} - \gamma$, each factor is at most $\sqrt{1 - 4\gamma^2}$, so the bound falls exponentially with the number of rounds. Freund and Schapire’s algorithm takes the initial distribution over the examples as an input; Viola and Jones’s choices are that distribution, which gives the two classes equal weight however unbalanced the training set is, and a weak learner restricted to one feature.

Detection and false-positive rates of the cascade

Let stage $i$ of a $K$-stage cascade have detection rate $d_i$ and false-positive rate $f_i$, each measured on the windows that reach that stage. A window is accepted only if every stage accepts it, so the cascade’s rates are the products

$$D = \prod_{i=1}^{K} d_i, \qquad F = \prod_{i=1}^{K} f_i.$$

This is the design rule of the cascade. The journal paper’s example: ten stages that each detect 99% of faces give $D = 0.99^{10} \approx 0.90$, and if each stage also passes 30% of the background, $F = 0.3^{10} \approx 6 \times 10^{-6}$. Each stage needs a very high detection rate but only a modest false-positive rate, which a small boosted classifier with a lowered threshold can deliver; the low overall false-positive rate comes from multiplication.

Expected cost per window

If stage $i$ has $n_i$ features and passes a fraction $p_i$ of the windows that reach it, the expected number of features evaluated per window is

$$\bar{n} = n_1 + \sum_{i=2}^{K} n_i \prod_{j=1}^{i-1} p_j ,$$

the journal paper’s expression written with stages numbered from 1. Since nearly all windows are background, $p_i$ is effectively the false-positive rate $f_i$, as the journal paper notes. With $f_i = 0.5$, stage $i$ is reached by $2^{-(i-1)}$ of the windows, so the late stages cost little on average however many features they hold. Viola and Jones considered choosing the number of stages, their sizes and their thresholds to minimize $\bar{n}$ for given $F$ and $D$, but called that optimization very difficult and used the per-stage targets instead.

Parameters

• Per-stage detection rate $d$. The overall detection rate is about $d^K$, so $d$ must be close to 1: 0.99 per stage over ten stages loses about one face in ten. Raising $d$ makes every stage larger, since a lower threshold lets more background through.
• Per-stage false-positive rate $f$. A larger $f$ (weaker stages) means more stages for the same $F_{\text{target}}$ but cheaper early stages. Lienhart, Kuranov and Pisarevsky used $f = 0.5$ and $d = 0.999$ over 20 stages, and pointed out that this gives an overall false-alarm rate of about $0.5^{20} \approx 10^{-6}$ and a hit rate of about $0.999^{20} \approx 0.98$ if their test set is representative; OpenCV’s training tool defaults to $f = 0.5$, $d = 0.995$ and 20 stages.
• Overall target $F_{\text{target}}$. Sets the number of stages; in the journal paper, Viola and Jones kept adding layers until the false-positive rate on their validation set was nearly zero while the detection rate stayed high.
• Base window size. $24 \times 24$ for Viola and Jones’s faces, which the journal paper says were cropped to include more of the head than earlier detectors used; its initial experiments with more tightly cropped $16 \times 16$ faces gave slightly worse results. Lienhart et al. found $20 \times 20$ best for frontal faces between 5 and 100 false alarms on the CMU test set, with $24 \times 24$ better below 5.
• Training set. Positives are often augmented: the conference paper adds mirror images of the training faces, and Lienhart et al. derived 5,000 positives from 1,000 faces by small random rotations, scalings, mirroring and shifts. The number of negatives per stage bounds what each stage can learn, and the face-free image collection bounds how many stages can be trained, since late stages need images whose windows still fool the cascade.
• Feature pool. The basic rectangle set, the extended set with rotated features, or other features altogether; the Haar-like features article covers the sets and their sizes.

Complexity

Training. In our accounting, sorting the examples once per feature costs $O(KN \log N)$; after that, finding the best stump over all features takes $O(KN)$ per round, because the example weights summarize everything earlier rounds did. The journal paper puts a 200-feature classifier at about $10^{11}$ operations with AdaBoost, against about $10^{16}$ for a wrapper method that re-evaluates every candidate feature together with the ones already chosen. Cascade training adds a scan of the face-free images after every stage. In the journal paper, Viola and Jones’s 38-layer face detector took on the order of weeks to train on a single 466 MHz AlphaStation XP900, and a parallelized version trained a complete cascade in about a day.

Detection. The cost of an image is the number of windows times $\bar{n}$ times the cost of a feature, 6 to 9 array references for the rectangle features, plus two integral images per image. In the conference paper an average of 10 of the cascade’s 6,061 features were evaluated per window on the MIT+CMU test set; the journal paper reports 8 of 6,060. The first stage’s two features cost about 60 microprocessor instructions per window by the authors’ count, and both versions report about 0.067 s per $384 \times 288$ image on a 700 MHz Pentium III. Memory is small: the integral images and a list of rectangles, thresholds and votes.

Implementation

The example builds a cascade in NumPy and measures what it buys. Images have intensities in $[0, 1]$. Faces are synthetic: a bright ellipse with two dark eye blocks and a dark mouth band, with random size, position, brightness and contrast, drawn on clutter of random rectangles, blobs, lines and face-like decoys (a bright ellipse with a single dark band). Training uses 800 faces, 800 validation faces, and 30 + 30 face-free $144 \times 144$ images scanned at stride 2 for negatives. The feature pool is 1,000 random two-, three- and four-rectangle features, each stored as weights on the corners of a window’s integral image and divided by the window’s standard deviation. Each stage is discrete AdaBoost over stumps as in Table 1 (in the code a stump with parity $p = 1$ accepts values above its threshold, the opposite sign convention to the formula above), with the threshold lowered from AdaBoost’s own where necessary so that at least 99% of the validation faces that reach the stage pass, and features are added until at most 50% of the validation negatives pass ($d = 0.99$, $f = 0.5$). Stages are added, each trained on 1,000 false positives of the cascade so far and validated on 1,000 more from the other images, until the product of the stage false-positive rates falls below $10^{-3}$. The test scans new $144 \times 144$ scenes with five faces each, all drawn at the $24 \times 24$ window size so that one scale and stride 1 suffice: detection is measured on the 500 windows placed exactly on the faces of 100 scenes, false positives on all windows of the first 16 scenes that do not overlap a face, and cost on those windows and on all windows of the 16 scenes. The script trains and tests five cascades, seeds 0 to 4, and runs in about 20 to 30 s.

import numpy as np

S = 24                                    # detection window: 24 x 24 pixels
yy, xx = np.mgrid[0:S, 0:S]

# ---------- synthetic images with intensities in [0, 1] ----------
def clutter(rng, H, W):
    """Gradient background with random rectangles, blobs, lines and face-like decoys
    (a bright ellipse with one dark band at a random height)."""
    y, x = np.mgrid[0:H, 0:W]
    base, gx, gy = rng.uniform(0.2, 0.6), *rng.uniform(-0.3, 0.3, 2)
    img = base + gx * x / W + gy * y / H
    for _ in range(H * W // 300):
        kind, cx, cy, v = rng.integers(4), rng.uniform(0, W), rng.uniform(0, H), rng.uniform()
        if kind == 0:
            w, h = rng.uniform(2, 20, 2); img[(abs(x - cx) < w / 2) & (abs(y - cy) < h / 2)] = v
        elif kind == 1:
            a, b = rng.uniform(3, 14, 2); img[((x - cx) / a) ** 2 + ((y - cy) / b) ** 2 <= 1] = v
        elif kind == 2:
            t = rng.uniform(0, np.pi)
            img[abs((x - cx) * np.sin(t) - (y - cy) * np.cos(t)) < rng.uniform(0.5, 2)] = v
        else:
            a, b = rng.uniform(7, 11), rng.uniform(9, 13)
            head = ((x - cx) / a) ** 2 + ((y - cy) / b) ** 2 <= 1
            img[head] = rng.uniform(0.55, 0.9)
            img[head & (abs(y - cy - rng.uniform(-b, b)) < rng.uniform(1, 3))] = rng.uniform(0, 0.4)
    return img

def draw_face(img, x0, y0, rng):
    """Face in the 24 x 24 window at (x0, y0): ellipse head, two dark eyes, dark mouth,
    with random size, position, skin brightness and contrast."""
    s, cx, cy, skin = rng.uniform(0.9, 1.1), 11.5 + rng.uniform(-1, 1), 12 + rng.uniform(-1, 1), rng.uniform(0.5, 0.9)
    p = img[y0:y0 + S, x0:x0 + S]
    p[((xx - cx) / (9.5 * s)) ** 2 + ((yy - cy) / (11.5 * s)) ** 2 <= 1] = skin
    ey, eye = cy - 3.5 * s + rng.uniform(-1, 1), skin - rng.uniform(0.3, 0.5)
    for ex in (cx - 4.5 * s, cx + 4.5 * s):
        p[(abs(xx - ex) < 2.5 * s) & (abs(yy - ey) < 1.5 * s)] = eye
    p[(abs(xx - cx) < 4 * s) & (abs(yy - cy - 6 * s - rng.uniform(-1, 1)) < s)] = skin - rng.uniform(0.2, 0.4)

def scene(rng, n_faces=0, H=144, W=144):
    img, boxes = clutter(rng, H, W), []
    while len(boxes) < n_faces:                       # faces at the window size, not overlapping
        x0, y0 = rng.integers(0, W - S + 1), rng.integers(0, H - S + 1)
        if all(max(abs(x0 - a), abs(y0 - b)) > S for a, b in boxes):
            draw_face(img, x0, y0, rng); boxes.append((x0, y0))
    return np.clip(img + rng.normal(0, 0.04, img.shape), 0, 1), boxes

def face_crop(rng):
    """A training face: drawn on 48 x 48 clutter, so that clutter reaches into the
    window as in a scene, then cropped to the 24 x 24 window."""
    img = clutter(rng, 2 * S, 2 * S)
    draw_face(img, S // 2, S // 2, rng)
    return np.clip(img + rng.normal(0, 0.04, img.shape), 0, 1)[S // 2:-S // 2, S // 2:-S // 2]

def face_mosaic(n, rng):
    """n training faces tiled into one image, one face per 24 x 24 tile."""
    img = np.vstack([np.hstack([face_crop(rng) for _ in range(20)]) for _ in range(n // 20)])
    ys, xs = np.mgrid[0:img.shape[0]:S, 0:img.shape[1]:S]
    return img, ys.ravel(), xs.ravel()

# ---------- Haar-like features on the integral image ----------
def feature_pool(K, rng):
    """K random two-, three- and four-rectangle features, as weights on the 25 x 25
    corner offsets of a window's integral image (four corners per rectangle)."""
    Wt = np.zeros(((S + 1) ** 2, K))
    for k in range(K):
        t = rng.integers(5)
        nx, ny = [(2, 1), (1, 2), (3, 1), (1, 3), (2, 2)][t]
        w, h = rng.integers(1, S // nx + 1), rng.integers(1, S // ny + 1)
        x, y = rng.integers(0, S - nx * w + 1), rng.integers(0, S - ny * h + 1)
        cells = {0: [(0, 0, -1), (1, 0, 1)], 1: [(0, 0, -1), (0, 1, 1)], 2: [(0, 0, -1), (1, 0, 1), (2, 0, -1)],
                 3: [(0, 0, -1), (0, 1, 1), (0, 2, -1)], 4: [(0, 0, 1), (1, 1, 1), (1, 0, -1), (0, 1, -1)]}[t]
        for i, j, sgn in cells:
            x0, y0 = x + i * w, y + j * h
            for cy, cx, c in ((y0 + h, x0 + w, 1), (y0, x0 + w, -1), (y0 + h, x0, -1), (y0, x0, 1)):
                Wt[cy * (S + 1) + cx, k] += sgn * c
    return Wt

def windows(img, ys=None, xs=None, stride=1):
    """Integral image, window corners and each window's standard deviation (from the
    integral images of I and I^2), used to normalize feature values."""
    I1 = np.pad(img, ((1, 0), (1, 0))).cumsum(0).cumsum(1)
    I2 = np.pad(img ** 2, ((1, 0), (1, 0))).cumsum(0).cumsum(1)
    if ys is None:
        ys, xs = [a.ravel() for a in np.mgrid[0:img.shape[0] - S + 1:stride, 0:img.shape[1] - S + 1:stride]]
    box = lambda I: (I[ys + S, xs + S] - I[ys, xs + S] - I[ys + S, xs] + I[ys, xs]) / S ** 2
    sd = np.sqrt(np.maximum(box(I2) - box(I1) ** 2, 1e-4))
    return I1, ys, xs, sd

oy, ox = [a.ravel() for a in np.mgrid[0:S + 1, 0:S + 1]]

def features(win, Wt, cols=None, keep=slice(None)):
    """Normalized values of the features `cols` (all if None) for windows `keep`."""
    I1, ys, xs, sd = win
    ys, xs, sd = ys[keep], xs[keep], sd[keep]
    if cols is None:                                   # all features: dense product
        return (I1[ys[:, None] + oy, xs[:, None] + ox] @ Wt) / sd[:, None]
    out = np.empty((len(ys), len(cols)))
    for c, k in enumerate(cols):                       # a few features: their corners only
        nz = np.nonzero(Wt[:, k])[0]
        out[:, c] = (I1[ys[:, None] + oy[nz], xs[:, None] + ox[nz]] @ Wt[nz, k]) / sd
    return out

# ---------- discrete AdaBoost over single-feature stumps (Viola and Jones, Table 1) ----------
def score(stumps, F):
    """Weighted vote of the stumps; F holds all K feature values per window."""
    return sum(a * (p * F[:, k] > p * th) for k, th, p, a in stumps)

def train_stage(F, y, Fv_pos, Fv_neg, d, f, n_max=60):
    """Add stumps until at most a fraction f of validation non-faces pass, with the
    threshold lowered until a fraction >= d of validation faces pass."""
    order = np.argsort(F, 0)
    w = np.where(y == 1, 1 / (2 * y.sum()), 1 / (2 * (1 - y).sum()))
    stumps = []
    while len(stumps) < n_max:
        w = w / w.sum()
        Sp = np.cumsum((w * y)[order], 0); Sn = np.cumsum((w * (1 - y))[order], 0)
        e1, e2 = Sp + (Sn[-1] - Sn), Sn + (Sp[-1] - Sp)   # error of each cut, both parities
        e = np.minimum(e1, e2)
        i, k = np.unravel_index(np.argmin(e), e.shape)
        v = F[order[:, k], k]
        th, p = (v[i] + v[min(i + 1, len(v) - 1)]) / 2, 1 if e1[i, k] <= e2[i, k] else -1
        beta = max(e[i, k], 1e-12) / (1 - e[i, k])
        h = p * F[:, k] > p * th
        w = w * beta ** (h == y)                      # correctly classified: weight times beta
        stumps.append((k, th, p, np.log(1 / beta)))
        sp, sn = score(stumps, Fv_pos), score(stumps, Fv_neg)
        thr = min(sum(s[3] for s in stumps) / 2, np.quantile(sp, 1 - d))
        if (sn >= thr).mean() <= f:
            break
    return {"stumps": stumps, "thr": thr}, (sp >= thr).mean(), (sn >= thr).mean()

def cascade(stages, win, Wt):
    """Run the cascade with early rejection; return the accepted mask, the number of
    features evaluated per window and the stage at which each window was rejected."""
    n = len(win[1])
    alive, nfeat, stop = np.ones(n, bool), np.zeros(n, int), np.full(n, len(stages))
    for s, st in enumerate(stages):
        idx = np.nonzero(alive)[0]
        G = features(win, Wt, [k for k, *_ in st["stumps"]], idx)   # this stage's features only
        sc = sum(a * (p * G[:, c] > p * th) for c, (k, th, p, a) in enumerate(st["stumps"]))
        nfeat[idx] += len(st["stumps"])
        alive[idx[sc < st["thr"]]] = False; stop[idx[sc < st["thr"]]] = s
    return alive, nfeat, stop

def false_positives(stages, wins, Wt, n, rng):
    """Up to n windows from face-free images that the current cascade accepts."""
    hits = [np.nonzero(cascade(stages, win, Wt)[0])[0] for win in wins]
    which = np.concatenate([np.full(len(h), i) for i, h in enumerate(hits)])
    pick = rng.permutation(len(which))[:n]
    allh = np.concatenate(hits)
    return np.vstack([features(wins[i], Wt, keep=allh[pick][which[pick] == i]) for i in range(len(wins))])

def train_cascade(rng, K=1000, n_face=800, n_neg=1000, d=0.99, f=0.5, F_target=1e-3):
    """Viola and Jones's Table 2 (IJCV 2004): add stages until the product of the stage
    false-positive rates on the validation non-faces falls below F_target."""
    Wt = feature_pool(K, rng)
    Fp, Fpv = features(windows(*face_mosaic(n_face, rng)), Wt), features(windows(*face_mosaic(n_face, rng)), Wt)
    wins_tr = [windows(scene(rng)[0], stride=2) for _ in range(30)]
    wins_val = [windows(scene(rng)[0], stride=2) for _ in range(30)]
    stages, log, F_prod = [], [], 1.0
    while F_prod > F_target:
        Fn, Fnv = false_positives(stages, wins_tr, Wt, n_neg, rng), false_positives(stages, wins_val, Wt, n_neg, rng)
        if min(len(Fn), len(Fnv)) < 100:
            break                                      # face-free images exhausted
        st, di, fi = train_stage(np.vstack([Fp, Fn]), np.r_[np.ones(len(Fp)), np.zeros(len(Fn))], Fpv, Fnv, d, f)
        stages.append(st); log.append((len(st["stumps"]), di, fi)); F_prod *= fi
        Fp = Fp[score(st["stumps"], Fp) >= st["thr"]]      # later stages see only the
        Fpv = Fpv[score(st["stumps"], Fpv) >= st["thr"]]   # faces that pass this one
    return stages, log, Wt

# ---------- test on new scenes: every window at one scale, stride 1 ----------
def test(stages, Wt, rng, n_scenes=100, n_scan=16, n_faces=5):
    """Stage reached by the windows placed exactly on the faces of n_scenes scenes,
    and by all windows of the first n_scan scenes that do not overlap a face,
    with the number of features evaluated per window over all their windows."""
    stop_face, stop_bg, nfeat, nfeat_bg = [], [], [], []
    for t in range(n_scenes):
        img, boxes = scene(rng, n_faces)
        x0, y0 = np.array(boxes).T
        stop_face.append(cascade(stages, windows(img, y0, x0), Wt)[2])
        if t < n_scan:
            win = windows(img)
            _, nf, stop = cascade(stages, win, Wt)
            dist = np.min([np.maximum(abs(win[2] - a), abs(win[1] - b)) for a, b in boxes], 0)
            stop_bg.append(stop[dist >= S]); nfeat.append(nf); nfeat_bg.append(nf[dist >= S])
    return [np.concatenate(a) for a in (stop_face, stop_bg, nfeat, nfeat_bg)]

for seed in range(5):
    rng = np.random.default_rng(seed)
    stages, log, Wt = train_cascade(rng)
    sf, sb, nf, nfb = test(stages, Wt, rng)
    n = np.array([l[0] for l in log]); d = np.array([l[1] for l in log]); f = np.array([l[2] for l in log])
    pred = n[0] + sum(n[i] * f[:i].prod() for i in range(1, len(n)))   # expected features per window
    if seed == 0:
        print("seed 0      validation        test windows")
        print("stage  n_i    d_i    f_i       d_i    f_i   background windows reaching the stage")
        for i in range(len(n)):
            dt = (sf > i).sum() / (sf >= i).sum(); ft = (sb > i).sum() / (sb >= i).sum()
            print(f"{i + 1:5d} {n[i]:4d}  {d[i]:.3f}  {f[i]:.3f}     {dt:.3f}  {ft:.3f}   {(sb >= i).sum():7d}")
        print("\n             features    per window         detection rate D   false-positive rate F")
        print("seed stages  in cascade  bg (pred)   all    prod d_i  test      prod f_i  test")
    print(f"{seed:4d} {len(n):6d} {n.sum():11d}  {nfb.mean():4.1f} ({pred:3.1f})  {nf.mean():4.1f}     "
          f"{d.prod():.3f}  {(sf == len(n)).mean():.3f}     {f.prod():.5f}  {(sb == len(n)).mean():.5f}")

Output:

seed 0      validation        test windows
stage  n_i    d_i    f_i       d_i    f_i   background windows reaching the stage
    1    3  0.996  0.431     1.000  0.441    112226
    2    3  0.996  0.489     0.994  0.476     49512
    3    3  0.992  0.468     0.974  0.457     23582
    4    5  0.990  0.338     0.986  0.344     10785
    5    6  0.990  0.448     0.987  0.440      3708
    6    4  0.995  0.424     0.987  0.446      1631
    7    7  0.990  0.413     0.976  0.415       727
    8    7  0.991  0.440     0.996  0.421       302
    9    1  0.993  0.496     0.993  0.535       127

             features    per window         detection rate D   false-positive rate F
seed stages  in cascade  bg (pred)   all    prod d_i  test      prod f_i  test
   0      9          39   5.8 (5.7)   6.1     0.935  0.898     0.00057  0.00061
   1      7          28   3.3 (3.3)   3.6     0.943  0.908     0.00121  0.00106
   2      8          52   6.4 (6.1)   6.8     0.929  0.916     0.00118  0.00120
   3      8          51   5.3 (5.3)   5.7     0.931  0.938     0.00084  0.00083
   4      9          50   5.7 (5.6)   6.2     0.929  0.918     0.00067  0.00056

The per-stage rates multiply as the formulas say. For seed 0, about 44% of the background windows survive each stage, and the count of windows reaching each stage shrinks from 112,226 to 127 at the ninth; across the five seeds, the product of the validation false-positive rates (0.00057 to 0.00121) matches the false-positive rate measured on the test scenes (0.00056 to 0.00120) to within about 20% for every seed. The test detection rates are 0.898 to 0.938 against products of 0.929 to 0.943. In our reading, the shortfall arises because each stage’s threshold was set on the validation faces, which makes the validation $d_i$ slightly optimistic: on the test faces of seed 0, five of the nine stages pass between 97.4% and 98.7% rather than at least 99%. With 500 test faces per seed, the test rate also carries a standard error of about 1.2 percentage points.

The cost column is the point of the cascade. The full cascades hold 28 to 52 features, the number every window would need without early rejection, while background windows evaluate 3.3 to 6.4 of them on average, and the expected cost $\bar{n}$ computed from the validation rates predicts this to within 0.3 features. Averaged over all windows, including those on and partly overlapping faces, which pass more stages, the cost rises to 3.6 to 6.8. Most windows are settled by the first three stages, with 3 features each in seed 0. The stage sizes grow irregularly rather than steadily, because each stage stops at the first feature that pushes its validation false-positive rate below 0.5, and in seed 0 a single feature was enough for the ninth.

Properties and Behavior

• Rates multiply only for the windows each stage sees. $D = \prod d_i$ and $F = \prod f_i$ hold exactly when each $d_i$ and $f_i$ is measured on the windows that reach stage $i$. A stage’s rates on fresh data differ from those on the data used to set its threshold; in the example the false-positive products transfer well, while the test detection rate falls up to about 4 points below the product in four of the five seeds.
• Each stage is harder than the last. Viola and Jones observed that the examples reaching deeper stages are harder than typical ones, which pushes each stage’s ROC curve down: at a given detection rate, a deeper stage has a higher false-positive rate. The face detector in the journal paper starts with stages of 2, 10, 25, 25, 50, 50 and 50 features, followed by stages of various sizes.
• Lowering thresholds leaves AdaBoost’s theory behind. The journal paper notes that it is not clear whether adjusting the threshold preserves AdaBoost’s training and generalization guarantees. A stage is trained to minimize error and then used at a different operating point; asymmetric boosting and later cascade methods (see Variants) address this.
• Accuracy is close to a single large classifier. In an experiment in the journal paper only, a cascade of ten 20-feature stages and a single 200-feature classifier trained on the union of the cascade’s examples had similar ROC curves, while the cascade was nearly 10 times faster. The authors also point out a hidden benefit: the late stages of a cascade effectively search hundreds of millions of negative windows to find a set of 10,000 hard ones to train on, where a single classifier must make do with a fixed set of perhaps 10,000 to 100,000.
• Reported accuracy. On the MIT+CMU frontal face test set of 130 images with 507 faces, the 38-layer detector’s detection rates in the tables of both versions range from 76.1% at 10 false detections to 93.9% at 167, and the journal table adds 94.1% at 422; the Viola–Jones paper article covers the comparison with earlier detectors.

The conference and journal versions differ in the details of this face detector, as the feature counts under Complexity show. The two-feature first stage passes 40% of the background in the conference paper and 50% in the journal paper, and the conference paper’s results section lists a one-feature first layer where its cascade section describes two features; the Viola–Jones paper article sets out these differences.

Limitations

• One viewpoint per cascade. A cascade learns one appearance at one aspect ratio. The journal paper reports that its frontal detector, informally, handled faces tilted up to about ±15° in the image plane and about ±45° out of plane, and became unreliable beyond that; other views need other cascades.
• Fixed, weak features. Rectangle features see only coarse contrasts in a few orientations. The journal paper names harsh backlighting and occluded eyes as failure modes of its face detector. For classes with articulated or variable shapes, such as pedestrians, later detectors turned to gradient features such as HOG and to part-based models such as deformable part models.
• Hand-set targets and slow training. The per-stage targets, the number of stages and the stage sizes are chosen by the user, and the result depends on them; training is slow because every stage requires a new search for false positives. Zhang and Viola (2007) list the number and shapes of the rectangle filters, the number of stages, the number of weak classifiers per stage and the per-stage detection targets as parameters that are hard to choose well.
• Errors only accumulate. A face rejected by any stage is lost; no later stage can recover it, and each stage’s small miss rate compounds over the cascade.
• Superseded in accuracy. Detectors based on convolutional neural networks, covered by the one-stage and two-stage detector articles, learn their features from data instead of choosing them from a fixed rectangle set.

Variants

• Asymmetric AdaBoost. In a NIPS 2001 paper, Viola and Jones argued that AdaBoost minimizes a bound on classification error, while a cascade stage needs few false negatives, and that the features AdaBoost selects are therefore not the best ones for rejecting background. Up-weighting the positives at the start does not help much, because AdaBoost’s reweighting absorbs the asymmetry in the first round. Their asymmetric AdaBoost spreads it over all $N$ rounds instead: before each round, each example’s weight is multiplied by $\exp\!\big(\frac{1}{N} y_i \log \sqrt{k}\big)$, with $y_i = \pm 1$ and false negatives costing $k$ times as much as false positives. They worked in the confidence-rated form of boosting of Schapire and Singer, in which each stump outputs a real value on each side of its threshold. With 4-feature classifiers, the false-positive rate fell by about 20% at 99% detection (their Figure 3), and in complete cascades on the MIT+CMU set the asymmetric version cut false positives by about half at a 91% detection rate.
• Gentle and Real AdaBoost, and small trees. Lienhart, Kuranov and Pisarevsky (2003) trained 20-stage cascades with Discrete, Real and Gentle AdaBoost on the basic feature set with stumps. At 10 false alarms on the CMU frontal face test set (130 images, 510 faces in their count), Gentle AdaBoost detected 82.7% of the faces, Discrete AdaBoost 79.5% and Real AdaBoost 75.4%, with the detector scaled in steps of 1.1; with steps of 1.2, Gentle AdaBoost fell to 68.8%. Their text calls Gentle AdaBoost the fastest of the three, but their Table 1, at a $20 \times 20$ pattern size, shows Real AdaBoost evaluating fewer features per background window with stumps, 26.28 against 30.99 for Gentle and 45.09 for Discrete. Replacing stumps with small CART trees of 2 to 4 splits improved detection at a similar cost. They also found that when the detector is scaled by non-integer factors, rounding the rectangle corners changes the ratio of the rectangle areas, and that re-weighting the rectangles to restore the original ratio improved results markedly.
• Rotated features. Lienhart and Maydt’s extended feature set adds 45° rectangle features; the Haar-like features article covers it and its reported gain.
• Soft cascades. Several methods relax the separate stages so that the scores of earlier weak classifiers count towards later decisions; Zhang and Viola (2007) name the boosting chain of Xiao, Zhu and Zhang, WaldBoost and the soft cascade among them. In the soft cascade of Bourdev and Brandt (2005), as Zhang and Viola describe it, the detector is a single boosted classifier with hundreds or thousands of weak classifiers, and evaluation stops as soon as the running sum falls below a rejection threshold set for that position in the sum. Zhang and Viola’s own multiple-instance pruning sets these thresholds after training from the full classifier’s decisions: positives the full classifier rejects may be dropped early, and since a face needs only one of its many acceptable windows to be detected, the thresholds need to keep only one such window for each face the full classifier detects.
• WaldBoost. Šochman and Matas (2005) treated detection as a sequential decision and used Wald’s sequential probability ratio test, which minimizes the average time to a decision for given error rates. AdaBoost selects and orders the measurements, the running sum $H_t(x)$ stands in for the likelihood ratio, and two thresholds per weak classifier, estimated from validation data, decide to accept, reject or continue. Only the overall error rates are specified, not per-stage rates, and samples are pruned and new negatives bootstrapped after every weak classifier rather than every stage. For faces they set the target false-positive rate to zero, so the test only ever rejects. On the MIT+CMU set, their 400-weak-classifier detector evaluated 10.84 weak classifiers per window on average; in their comparison table, Viola and Jones’s detector, at 8, was the only faster method, with worse detection rates.
• OpenCV. As of October 2026, OpenCV 4.14.0 implements the cascade in cv::CascadeClassifier, whose detectMultiScale shrinks the image by a default factor of 1.1 per scale and keeps a candidate only if it has at least 3 neighbouring detections (both defaults can be changed). The source of its training tool, opencv_traincascade, supports Haar-like (the default), LBP and HOG features and Discrete, Real, Logit and Gentle AdaBoost, with Gentle AdaBoost as the default; other defaults are a stage hit rate of 0.995, a stage false-alarm rate of 0.5, 20 stages, stumps (tree depth 1), at most 100 weak classifiers per stage and weight trimming at 0.95. Each stage’s threshold is set so that the minimum hit rate of the training positives pass, weak classifiers are added until the fraction of training negatives that pass falls to the false-alarm rate, and training stops early once the fraction of negative windows that pass the whole cascade falls below $0.5^{20}$ (with the defaults). The bundled haarcascade_frontalface_default.xml, described in its header as a stump-based $24 \times 24$ “discrete(?)” AdaBoost detector by Rainer Lienhart, has 25 stages and 2,913 weak classifiers, from 9 in the first stage to at most 211 in a later one; haarcascade_frontalface_alt.xml, a stump-based $20 \times 20$ Gentle AdaBoost detector, has 22 stages and 2,135. The Haar-like features article covers the training tool’s build status in 4.14.0 and the move of the cascade detector to opencv_contrib in OpenCV 5.0.0.