Labeled Faces in the Wild

Labeled Faces in the Wild (LFW) is a dataset of 13,233 photographs of the faces of 5,749 people, collected from news images on the web and released in 2007 by Gary B. Huang, Manu Ramesh, Tamara Berg and Erik Learned-Miller at the University of Massachusetts Amherst, in a technical report dated October 2007. It defines one task: given two face images, decide whether they show the same person. The 2007 report calls this pair matching; the curators adopted the now usual name, face verification, in their 2014 update (Huang & Learned-Miller, 2014). Every image is the output of a Viola–Jones face detector, the test people never appear in training, and results are reported as a mean accuracy over ten fixed folds with its standard error. From 2008 to about 2015 it was the main benchmark on which face recognition moved from hand-designed descriptors to deep networks, and it has been saturated since: the best listed accuracies exceed 99.8%.

Purpose

The 2008 workshop paper (Huang et al., 2008) contrasts LFW with labelled face databases such as the Face Recognition Grand Challenge, BioID, FERET and CMU PIE, which were typically captured under controlled conditions, with fewer people and less diversity (§2). LFW was built instead from pre-existing photographs taken for other purposes, to approximate the “natural” variation of everyday images (Huang et al., 2007, §II).

It targets a specific form of the problem, which the report calls the unseen pair match problem: for any training and test split the people are mutually exclusive, so no model of a test person can be built in advance, and each test pair must be decided independently, as if it were the only pair. The second condition rules out, for example, semi-supervised methods that look across the whole test set (§III.A). The authors also designed it to fit a detection–alignment–recognition pipeline: because every face is a Viola–Jones detection, an alignment or recognition method that works on LFW should work after that detector in a complete system (§V).

Collection

The raw images came from the Berkeley “Faces in the Wild” collection (Huang et al., 2007, §VI.A): about half a million news photographs with captions gathered from Yahoo News over roughly two years, from which Berg, Berg, Edwards and Forsyth (2004) extracted faces and linked them automatically to names in the captions, with a labelling accuracy of 77%. LFW’s report gives that figure and adds that the collection was not meant as a benchmark: it had many label errors and duplicates, and researchers had cut ad hoc subsets from it (§II).

The 2007 report describes each step (§VI):

1. Detection. OpenCV 1.0.0’s Viola–Jones detector, cvHaarDetectObjects with the cascade haarcascade_frontalface_default.xml, scale factor 1.2, minimum neighbours 2 and Canny pruning, was run on every image.
2. Screening. An operator removed each detection that was not a face, and each face whose person could not be named from general knowledge or the caption.
3. Duplicates. Images judged to come from a common source photograph were removed, however they had been recropped, rescaled or recompressed; distinct photographs taken nearly simultaneously were kept. The report says some duplicates may remain.
4. Naming. Names were assigned by hand with the caption as an aid, one name per person (“Bob McNamara” and “Robert McNamara” merged, Chinese names kept in family-name-first order), so a wrong caption could produce a wrong name.
5. Cropping. The detector’s box was enlarged by 2.2 in each dimension, padded with black pixels where it left the photograph, and resized to $250 \times 250$ pixels.

An image may contain several faces; the one containing the image’s central pixel is the labelled face and the others are background. The authors state the resulting biases themselves: few images in very poor or extreme lighting and, because the detector acted as a filter, few side views and few views from above or below (§II).

Dataset Size

The official page lists 13,233 images, 5,749 people and 1,680 people with two or more images; the report adds that the other 4,069 people have one image each, 2.30 images per person on average (Huang et al., 2007, §I–II). Most images are in colour, a few greyscale. The ECCV 2008 workshop paper tabulates the long tail (Table 1b):

Images per person	People	Images
1	4,069 (70.8%)	4,096 (30.7%)
2–5	1,369 (23.8%)	3,739 (28.3%)
6–10	168	1,251
11–20	86	1,251
21–30	25	613
31–80	27	1,170
> 81 (as printed)	5	1,140

The single-image row prints 4,096 images for 4,069 people, which cannot both be right. With 4,069 in its place the image column sums to 13,233, and 30.7% of 13,233 is 4,069, so the 4,096 is presumably a typographical error (our reading). The README’s file-format example lists George W. Bush with 530 images, the person with the most images according to the official page.

Classes

The classes are the 5,749 people, each identified by a name that serves as the directory and file name (George_W_Bush/George_W_Bush_0010.jpg). The benchmark itself has two classes of pairs: matched (same person) and mismatched.

Annotation Types

Each image carries one label, the person’s name, and the benchmark files add the pair labels. The official page offers three aligned versions alongside the original crops, all for the same images:

• Funneled images, aligned by the congealing method of Huang, Jain and Learned-Miller (2007), whose distribution fields were learned from one image each of 800 randomly chosen people (workshop paper, §6.1).
• LFW-a, aligned with commercial software by Taigman, Wolf and Hassner, which the page describes as an unpublished method; the 2016 survey by the curators describes it as a similarity transform of seven fiducial points to fixed locations (Learned-Miller et al., 2016, §3).
• Deep funneled images, from congealing with features learned without labels by convolutional restricted Boltzmann machines and convolutional deep belief networks (Huang et al., 2012), added to the site in August 2013.

The page reports that LFW-a and the deep funneled images give better results than the original or funneled images for most verification algorithms. It also links superpixel segmentations, attribute values (Kumar et al., 2009) and part labels and, among resources it says it has not verified, gender labels and frontalized images (LFW3D).

Errata. The page lists known labelling errors and asks users to keep the data as released, because earlier results did not have the corrections. Its text says six matched pairs in View 2 are wrongly labelled, but its list, last updated in January 2018, gives seven: three in fold 1, three in fold 5 and one in fold 7. Other errors include images of the wrong person, duplicates filed under different names and people listed under two names (Wang Nan and Nan Wang).

Splits

The data are organized into two views (Huang et al., 2007, §III.B, §VI.F; README):

• View 1, for development. People were assigned to a training set with probability 0.7, the rest to a test set. pairsDevTrain.txt holds 1,100 matched and 1,100 mismatched pairs and pairsDevTest.txt 500 of each; peopleDevTrain.txt and peopleDevTest.txt list the people for the unrestricted setting. Models, features and parameters may be tried on it as often as wanted.
• View 2, for reporting. People were assigned uniformly at random to ten sets, and 300 matched and 300 mismatched pairs were drawn within each, 6,000 pairs in all, listed in pairs.txt; people.txt lists each set’s people. Because people are split before pairs are drawn, no person appears in two sets.

Matched pairs were drawn by choosing a person with at least two images uniformly at random, regardless of how many images that person has, then two of their images; mismatched pairs by choosing two people of the set and one image of each; repeated pairs were redrawn (§VI.F). The two views were drawn independently from the whole database, so images of View 1 can reappear in View 2. The report accepts the small bias this introduces for larger training and test sets, and warns that a classifier which memorized View 1 could score artificially well on View 2; it argues that such a classifier would be caught by its poor score on View 1’s disjoint test set (§III.B).

The workshop paper also defined an identification protocol, matching a probe image to a gallery of known people, which it called face verification, the name the 2014 update later gave to pair matching: the 158 people with at least ten images, 38 of them (892 images) for development and 120 (3,432 images) split ten times, 90% of each person’s images for training and 10% for testing, reported as micro- and macro-averaged accuracy. Eigenfaces reached 0.1270 ± 0.0045 there (Table 2). Neither the technical report nor the official page includes this protocol.

Tasks

LFW is a benchmark for face verification of single still images, the one-to-one comparison defined in face recognition. It has also served as data for face alignment (the three aligned versions), for attribute prediction (Kumar et al.’s attribute values, and LFWA in the CelebA paper) and, through protocols defined by others, for identification (BLUFR, below).

Evaluation

Training paradigms (2007)

The 2007 report asks every result to state which of two training paradigms it used (§IV):

• Image-restricted. Only the given pairs and their matched/mismatched labels may be used; names may identify files but must not be given to the algorithm. If pairs (10, 12) and (42, 50) of one person are given, the pair (10, 42) may not be added, because only the name links them. Augmenting by image identity is allowed: if (1, 2) and (2, 3) are matched pairs, (1, 3) may be added.
• Unrestricted. The names of the people in the training sets may be used to form as many matched and mismatched training pairs as wanted. Test pairs are always those of pairs.txt.

The two documents differ on one detail. The report says mismatched training pairs should combine people from the same View 2 set; the README says that under the unrestricted configuration they may combine images from different training sets.

Outside data: the six categories of 2014

As researchers began training on images, labels and tools from outside LFW, the curators first split the image-restricted results by use of outside data and then, in May 2014, replaced the two paradigms with six protocols, which the update asks authors to name exactly (Huang & Learned-Miller, 2014, Table I; its separate columns for outside images and outside annotations, and for outside pair labels and outside identities, have equal entries in every row and are merged here):

Protocol	LFW pair labels	LFW identities	Annotations of LFW training data	Outside images and annotations	Outside pair labels or identities
Unsupervised	no	no	yes	yes	no
Image-restricted, no outside data	yes	no	no	no	no
Unrestricted, no outside data	yes	yes	no	no	no
Image-restricted, label-free outside data	yes	no	yes	yes	no
Unrestricted, label-free outside data	yes	yes	yes	yes	no
Unrestricted, labeled outside data	yes	yes	yes	yes	yes

The rules behind the table:

• No outside data excludes eye detectors, alignment methods and feature extractors trained on outside data. Unsupervised tools trained only on LFW’s training sets are allowed, so the funneled and deep funneled images may be used and LFW-a may not. Generic descriptors such as SIFT, developed without regard to faces, are allowed; the curators reserve the judgement for descriptors adapted to faces.
• Label-free outside data may include face images without identities and annotations such as landmark positions, but nothing that could form same/different pairs. Videos of faces are excluded even unlabelled, because tracking a face yields matched pairs.
• Labeled outside data, the most permissive protocol, allows outside pairs, names and arbitrary annotations, provided they include no person in the LFW test set. There is no image-restricted version, because with arbitrary labelled outside data the distinction is no longer useful.
• Under every protocol, images of a person in the test set may not be used for training, and no protocol allows a human in the loop, such as hand-marked landmarks or manual alignment of test images.

When a paper does not make its category clear, the curators contact the authors, list the result under the most permissive category, or do not list it.

The ten-fold procedure

Reporting uses View 2 as ten experiments in a leave-one-set-out scheme (Huang et al., 2007, §III.B):

1. For experiment $i$, combine the nine other sets into a training set.
2. Set every parameter of the model chosen on View 1, including the decision threshold, using that training set only, giving classifier $i$.
3. Classify each pair of set $i$ from that pair alone, and record the percentage $p_i$ classified correctly.

At a minimum, authors report the estimated mean accuracy and the standard error of the mean:

$$\hat{\mu} = \frac{1}{10} \sum_{i=1}^{10} p_i, \qquad \hat{\sigma} = \sqrt{\frac{\sum_{i=1}^{10} (p_i - \hat{\mu})^2}{9}}, \qquad \mathrm{SE} = \frac{\hat{\sigma}}{\sqrt{10}} .$$

Tuning on the combined training sets of all ten experiments is forbidden, because each set is another experiment’s test set, and so is choosing a threshold after looking at test results, for example the point on a precision–recall curve with the highest accuracy. The report notes that because the training sets overlap, the SE may be biased downward, a general property of cross-validation, and adds in a footnote that overlapping SEs imply no significant difference at the 0.05 level, while non-overlapping SEs do not in general imply a significant one. The workshop version is slightly looser than the report on one point: it allows each experiment’s parameters to be set “using either the data in View 1 or only the training data”, and its Nowak baseline took its threshold from View 1 (§4, §7).

The results page also shows ROC curves for each category, plotted from a text file of (true positive rate, false positive rate) points per method, each point being the average over the ten folds of the two rates at one fixed threshold; the curves themselves are defined in ROC curve and AUC.

The unsupervised protocol

Defined in the 2014 update, the unsupervised protocol scores a scalar dissimilarity $f(I, J)$ between two images, calling a pair “same” when $f \le \theta$. Nothing about the labels may be used: not pair labels, not names, and not even the fact that half the training pairs are matched, which rules out setting $\theta$ by splitting the training distances in half. The threshold may not be set from training or test results at all; statistics of individual LFW training images, or of outside images without identity information, are allowed. With no legitimate threshold, the protocol reports the area under the ROC curve: sort the test pairs by $f$, use every value as a threshold plus one below all of them, and integrate. The results page provides lfw_auc.m, which reads the fold-averaged curve, sorts its points by false positive rate and applies the trapezoid rule (read line by line in the archived copy; the copies of 2014 and 2024 are identical).

Human performance

The results page lists human accuracy measured on Amazon Mechanical Turk in the study of Kumar et al. (2009): 99.20% on the funneled images, 97.53% on tight crops of the face and 94.27% in an “inverse mask” condition. It adds that the experiment did not control for whether the workers already knew the many famous people in the test sets, which amounts to labelled training data for test identities, so the numbers may overestimate humans on unfamiliar faces and fit none of the protocols.

Example: the View 2 protocol on rendered faces

The script builds a synthetic LFW from rendered geometric faces, not real ones: each of 1,500 identities is nine parameters (head size, eye position and size, brow gap, nose length, mouth width and height), rendered in $24 \times 24$ greyscale under small random rotation, scale and shift, a varying smile and eye opening, simulated sunglasses on 10% of images, a light ramp, gain and noise. As in LFW, 70% of identities have one image and the others a long-tailed number; people are split into ten sets, and 300 matched and 300 mismatched pairs per set are drawn by the report’s procedure. The classifier is an eigenface baseline in the spirit of the 2008 paper’s: principal components keeping 80% of the variance, fitted only on images that occur in training pairs (image-restricted, no outside data), Euclidean distance between projections, and an accuracy-maximizing threshold. Each experiment refits both on its nine training sets. The script also tunes the threshold on the test fold, which the protocol forbids, repeats everything on 30 independent synthetic datasets, and computes the unsupervised AUC of raw pixel distances as lfw_auc.m does. NumPy 2.3.5 and scikit-learn 1.7.2; about 20 s; two runs gave identical output.

import numpy as np
from sklearn.metrics import roc_auc_score

H = 24                                                   # image side in pixels
yy, xx = np.mgrid[:H, :H] - (H - 1) / 2

def render(p, rng):
    """One HxH image per row of p (identity parameters in [0, 1]): ellipse head,
    eyes, brows, nose and mouth, under random pose, expression, light and noise."""
    n = len(p)
    U = lambda lo, hi: rng.uniform(lo, hi, (n, 1, 1))
    P = lambda k, lo, hi: (lo + (hi - lo) * p[:, k])[:, None, None]
    th, s = np.deg2rad(U(-5, 5)), U(0.96, 1.04)            # in-plane rotation, scale
    dx, dy = U(-0.8, 0.8), U(-0.8, 0.8)                    # residual detector shift
    x = ((xx - dx) * np.cos(th) + (yy - dy) * np.sin(th)) / s
    y = (-(xx - dx) * np.sin(th) + (yy - dy) * np.cos(th)) / s
    a, b = P(0, 6.5, 10), P(1, 8, 11)                      # head half-width, half-height
    ex, ey, er = P(2, 2.5, 5), P(3, -5, -1.5), P(4, 0.9, 2)
    by, nl = P(5, 1.2, 2.8), P(6, 1, 4.5)                  # brow gap, nose length
    mw, my = P(7, 2, 5.5) * U(0.8, 1.2), P(8, 3, 6.5)      # mouth width (smile), height
    head = (x / a) ** 2 + (y / b) ** 2 < 1
    eyes = ((np.abs(x) - ex) / er) ** 2 + ((y - ey) / (er * U(0.4, 1.0))) ** 2 < 1
    brows = (np.abs(np.abs(x) - ex) < er + 0.5) & (np.abs(y - (ey - by - er)) < 0.5)
    nose = (np.abs(x) < 0.6) & (y > ey + 1) & (y < ey + 1 + nl)
    mouth = (np.abs(x) < mw) & (np.abs(y - my) < 0.8)
    shades = (rng.random((n, 1, 1)) < 0.1) & (np.abs(y - ey) < 1.6) & (np.abs(x) < a)
    img = U(0.1, 0.4) + head * U(0.4, 0.6) - (eyes | brows | nose | mouth | shades) * head * U(0.3, 0.5)
    img = (img + U(-0.02, 0.02) * xx + U(-0.02, 0.02) * yy) * U(0.7, 1.3)  # light ramp, gain
    img = img + 0.05 * rng.standard_normal((n, H, H))                      # sensor noise
    img = img.reshape(n, -1)
    return (img - img.mean(1, keepdims=True)) / img.std(1, keepdims=True)

def make_view2(rng, n_people=1500, n_sets=10, n_pairs=300):
    """A synthetic LFW: long-tailed images per person, people split uniformly at
    random into ten sets, then 300 matched and 300 mismatched pairs per set, drawn
    as the 2007 report describes (a person with >= 2 images uniformly, then two images)."""
    counts = np.where(rng.random(n_people) < 0.7, 1, np.minimum(2 + rng.geometric(0.25, n_people), 60))
    person = np.repeat(np.arange(n_people), counts)
    X = render(rng.random((n_people, 9))[person], rng)
    first = np.r_[0, np.cumsum(counts)[:-1]]                # index of each person's image 0
    set_of = rng.integers(n_sets, size=n_people)
    pairs, same, fold = [], [], []
    for k in range(n_sets):
        people = np.flatnonzero(set_of == k)
        multi = people[counts[people] >= 2]
        seen = set()
        while len(seen) < n_pairs:                           # matched pairs
            q = rng.choice(multi)
            i, j = sorted(rng.choice(counts[q], 2, replace=False))
            seen.add((first[q] + i, first[q] + j))
        pairs += sorted(seen); same += [1] * n_pairs
        seen = set()
        while len(seen) < n_pairs:                           # mismatched pairs
            q, r = rng.choice(people, 2, replace=False)
            seen.add((first[q] + rng.integers(counts[q]), first[r] + rng.integers(counts[r])))
        pairs += sorted(seen); same += [0] * n_pairs
        fold += [k] * (2 * n_pairs)
    return X, np.array(pairs), np.array(same), np.array(fold), len(person)

def eigenface_scores(X, pairs, train_pairs, var=0.8):
    """PCA fitted only on images that occur in training pairs (image-restricted,
    no outside data); score = minus the Euclidean distance between projections."""
    T = X[np.unique(pairs[train_pairs])]
    mu = T.mean(0)
    w, V = np.linalg.eigh(np.cov(T - mu, rowvar=False))
    w, V = w[::-1], V[:, ::-1]
    d = np.searchsorted(np.cumsum(w) / w.sum(), var) + 1    # keep 80% of the variance
    Z = (X - mu) @ V[:, :d]
    return -np.linalg.norm(Z[pairs[:, 0]] - Z[pairs[:, 1]], axis=1)

def best_threshold(s, y):
    """Threshold (midpoint between sorted scores) maximizing accuracy of s >= t."""
    o = np.argsort(s); s, y = s[o], y[o]
    acc = (np.r_[0, np.cumsum(1 - y)] + np.r_[np.cumsum(y[::-1])[::-1], 0]) / len(y)
    k = np.argmax(acc)                                      # first k scores called "different"
    return -np.inf if k == 0 else np.inf if k == len(s) else (s[k - 1] + s[k]) / 2

def view2(X, pairs, same, fold):
    """Ten experiments: train on nine sets, test on the tenth. Returns per-fold accuracy
    with the threshold set on the training folds, and with it set on the test fold."""
    honest, peeked = [], []
    for k in range(10):
        tr, te = fold != k, fold == k
        s = eigenface_scores(X, pairs, tr)
        t = best_threshold(s[tr], same[tr])                  # allowed: training folds only
        honest.append(np.mean((s[te] >= t) == same[te]))
        t_test = best_threshold(s[te], same[te])             # forbidden: tuned on the test fold
        peeked.append(np.mean((s[te] >= t_test) == same[te]))
    return np.array(honest), np.array(peeked)

def mean_se(p):
    return p.mean(), p.std(ddof=1) / np.sqrt(len(p))        # 2007 report: sigma-hat / sqrt(10)

def lfw_auc(f, same, fold):
    """Unsupervised protocol: fold-averaged (FPR, TPR) at every observed threshold,
    then the trapezoid rule, as in the official lfw_auc.m."""
    th = np.r_[np.inf, np.sort(np.unique(f))[::-1]]          # first threshold: all "different"
    tpr = np.mean([[(f[(fold == k) & (same == 1)] >= t).mean() for t in th] for k in range(10)], 0)
    fpr = np.mean([[(f[(fold == k) & (same == 0)] >= t).mean() for t in th] for k in range(10)], 0)
    o = np.argsort(fpr, kind="stable")
    return np.trapezoid(tpr[o], fpr[o])

rng = np.random.default_rng(0)
X, pairs, same, fold, n_img = make_view2(rng)
print(f"seed 0: {n_img} images, {len(pairs)} pairs in 10 sets of 300 + 300")
honest, peeked = view2(X, pairs, same, fold)
print("fold accuracy, threshold from 9 training folds:", np.round(honest, 3))
print("mean +- SE, threshold from training folds: %.4f +- %.4f" % mean_se(honest))
print("mean +- SE, threshold tuned on test fold:  %.4f +- %.4f" % mean_se(peeked))
f = -np.linalg.norm(X[pairs[:, 0]] - X[pairs[:, 1]], axis=1)  # no training at all: raw pixels
print("unsupervised AUC: fold-averaged ROC %.4f, pooled scikit-learn %.4f"
      % (lfw_auc(f, same, fold), roc_auc_score(same, f)))

rows = []
for seed in range(1, 31):                                    # 30 independent synthetic LFWs
    X, pairs, same, fold, _ = make_view2(np.random.default_rng(seed))
    honest, peeked = view2(X, pairs, same, fold)
    rows.append((*mean_se(honest), peeked.mean()))
m, se, pk = np.array(rows).T
print("\nseeds 1-30: mean accuracy %.4f to %.4f" % (m.min(), m.max()))
print("SD of the mean across datasets %.4f; reported SE %.4f to %.4f (average %.4f)"
      % (m.std(ddof=1), se.min(), se.max(), se.mean()))
print("gain from tuning on the test fold: %.4f to %.4f (average %.4f)"
      % ((pk - m).min(), (pk - m).max(), (pk - m).mean()))

seed 0: 4040 images, 6000 pairs in 10 sets of 300 + 300
fold accuracy, threshold from 9 training folds: [0.62  0.607 0.647 0.648 0.63  0.662 0.635 0.647 0.653 0.648]
mean +- SE, threshold from training folds: 0.6397 +- 0.0053
mean +- SE, threshold tuned on test fold:  0.6538 +- 0.0054
unsupervised AUC: fold-averaged ROC 0.6902, pooled scikit-learn 0.6902

seeds 1-30: mean accuracy 0.6130 to 0.6475
SD of the mean across datasets 0.0082; reported SE 0.0036 to 0.0087 (average 0.0063)
gain from tuning on the test fold: 0.0073 to 0.0238 (average 0.0132)

The fold accuracies spread over five and a half points although the ten experiments run the same procedure, which is why the protocol asks for the standard error and not only the mean. Choosing the threshold on the test fold raised the mean in every one of the 30 datasets, by 0.7 to 2.4 points, on average about twice the average reported SE: an advantage obtained without any change to the classifier. The protocol’s ban on test-fold thresholds is aimed at exactly this.

The SE question can be asked of synthetic data because new datasets can be drawn. Across the 30 datasets the ten-fold mean varied with a standard deviation of 0.0082, while the SE each dataset reported averaged 0.0063, about a quarter lower, the direction the 2007 report warns about. With 30 datasets the standard deviation is itself uncertain by roughly 13% (for a normal sample its relative standard error is about $1/\sqrt{2 \cdot 29}$), so this is supporting evidence, not a measurement of the bias. The two AUCs agree exactly: by our reasoning, because every fold has 300 matched and 300 mismatched pairs, averaging the ten folds’ rates at a common threshold gives the pooled rates, so LFW’s fold-averaged ROC is the pooled ROC.

License

As of October 2026 the official host, vis-www.cs.umass.edu, does not resolve, and this article relies on archived copies; the latest capture of the home page is from 5 January 2025. Neither that page nor the README or the results page states a license or terms of use for the images, the pair files or the aligned versions; the page asks users to cite the 2007 report and, for the funneled and deep funneled images, the alignment papers. The images are news photographs gathered from Yahoo News, and the pages say nothing about their copyright. The terms are therefore unclear.

Limitations and Bias

• Detector-filtered faces. Drawing on news photographs and keeping only frontal-face detections limited pose and lighting, as the authors note. The later IARPA Janus benchmarks localized faces by hand for this reason, as the 2016 survey points out for IJB-A.
• Easy negatives and few of them. Mismatched pairs join two randomly chosen people, who often look quite different; the curators’ survey calls verification “a problem in which many examples are easy” (Learned-Miller et al., 2016, §4.1). View 2 has 3,000 mismatched pairs, so a false accept rate of 0.1% would rest on three of them, which led Liao et al. (2014) to call low-FAR evaluation on LFW statistically unsound. NIST describes LFW with its essentially equal-error-rate metric as an easy task, verification at very high false match rates (Grother, Ngan & Hanaoka, 2019).
• A ceiling below 100%. The survey counts five labelling errors among FaceNet’s 22 errors, so a perfect classifier would score about 5,995/6,000, or 99.9%, and a method scoring higher would have to “miss” mislabelled pairs, which it suggests is a sign of fitting to the test data (§4). The survey was written before the errata updates of 2017 and 2018, each of which added a wrongly labelled matched pair in View 2.
• Familiar faces. The people are public figures from the news, so human judges may already know them, the point the results page makes about the human experiment. For models, the labeled-outside-data protocol forbids training on test-set people; in our reading, that is hard to check for training sets of millions of web images of celebrities.
• Who is in it, and what it can show. Since September 2019 (the disclaimer appears in a capture of 26 September 2019 and not in one of 6 September), the official page has warned that no LFW result should be used to conclude that an algorithm is suitable for commercial use. Its reasons: verification and identification are very different problems; many groups are poorly represented, the demographic gaps summarized in dataset bias; the dataset is too small for strong conclusions about subgroups; and poor lighting, extreme pose, strong occlusion and low resolution are not a major part of it. It points to NIST’s vendor tests for assessing commercial systems. The dataset bias article also follows how LFW’s use shifted towards commercial systems.

Common Uses

LFW was the standard verification benchmark for papers on face descriptors, metric learning and alignment from 2008 and for the deep networks of 2014–2015. Training sets were built around it: the description of CASIA-WebFace that LFW’s page quotes encourages methods trained on it to report on LFW, and CelebFaces+, the image source of CelebA, has no person in common with LFW (Sun, Wang & Tang, 2014). Later papers still report LFW accuracy next to harder benchmarks, often above 99%, where differences of a few pairs are within the cautions of benchmark overfitting; ArcFace gives examples.

Historical Milestones

Accuracies are $\hat{\mu} \pm \mathrm{SE}$ on View 2 as listed on the archived results page (December 2024), in its 2014 categories.

Year	Result	Protocol category	Source
2007–2008	Eigenfaces 0.6002 ± 0.0079; Nowak and Jurie’s method, run by the curators, 0.7245 ± 0.0040 (original images), 0.7393 ± 0.0049 (funneled)	Image-restricted, no outside data	Results page; workshop paper Table 2 prints 0.7333 ± 0.0060 for the funneled run
2009	The same Nowak method on LFW-a: 0.7912	Image-restricted, on LFW-a (outside data)	Results page, introduction (Wolf et al.)
2009	Humans on Mechanical Turk: 0.9920 (funneled), 0.9753 (cropped)	Fits none (see above)	Results page, Kumar et al.
2013	High-dimensional LBP: 0.9517 ± 0.0113	Unrestricted, labeled outside data	Results page
2014	DeepFace ensemble: 0.9735 ± 0.0025	Unrestricted, labeled outside data	Results page
2014	DeepID2: 0.9915 ± 0.0013	Unrestricted, labeled outside data	Results page, which cites the arXiv preprint and marks the entry as a commercial system
2015	FaceNet: 0.9963 ± 0.0009	Unrestricted, labeled outside data	Results page; survey, §4
to 2024	Best listed entry, a commercial system: 0.9988 ± 0.0004	Unrestricted, labeled outside data	Results page

DeepFace came within two tenths of a point of the cropped-face human figure, the gap of its title, and FaceNet’s 99.63% was, as the survey was written, the best result described in a peer-reviewed paper, leaving 22 errors. The rest of the climb came mostly from commercial systems, which the page marks as unpublished and not peer-reviewed and which, it says, researchers need not compare against; from June 2020 it posted only results exceeding the best in their category. In 2019 the PAMI Technical Committee gave LFW the Mark Everingham Prize, “for generating and maintaining the LFW dataset and benchmark, starting from 2007” (PAMI TC recipients list).

Related Datasets

• BLUFR (Liao et al., 2014) re-uses all 13,233 images for verification and open-set identification at low false accept rates: ten random trials, each training on 1,500 people and testing on the remaining 4,249 (the text’s figure; Table 1 prints 4,294), with about 157,000 genuine and 47 million impostor scores per trial, an open-set gallery of 1,000 people with one image each, and results reported as mean minus standard deviation over trials. The best of its 21 baselines, high-dimensional LBP with Joint Bayesian, which reaches 93.18% under LFW’s unrestricted protocol, verified 41.66% of genuine pairs at a false accept rate of 0.1%.
• Fine-grained LFW (FGLFW), listed among the page’s resources, replaces the random mismatched pairs with 3,000 pairs of similar-looking people selected by crowdsourcing.
• YouTube Faces carries LFW’s ten-fold pair protocol to videos; IJB-A, IJB-B and IJB-C moved to hand-localized faces, templates and low-FAR operating points; MegaFace added a million distractors; MS-Celeb-1M, a training set, was withdrawn. Face recognition describes these benchmarks and dataset bias the withdrawal.
• FDDB, the face detection benchmark from the same group, linked from LFW’s page.

Resources

• LFW home page (archived): views, downloads, errata, disclaimer and change history.
• Results page (archived): results by category, ROC files and plotting scripts, lfw_auc.m.
• README (archived): training configurations and the formats of pairs.txt and people.txt.
• The 2007 technical report and the 2014 update (archived), and the ECCV 2008 workshop paper on HAL.
• LFW survey, the curators’ 2016 review of results and alignment methods (authors’ draft).