FaceNet

FaceNet is a face recognition model from Google, introduced by Florian Schroff, Dmitry Kalenichenko and James Philbin in 2015 (arXiv in March, CVPR in June). A deep convolutional network maps a cropped face to a 128-dimensional vector of unit length, an embedding in which the squared Euclidean distance between two faces measures how likely they are to show the same person. The network is trained for that comparison directly, with a triplet loss on an anchor face, a positive of the same identity and a negative of another, and with negatives chosen online inside each mini-batch by a rule the authors called “semi-hard”. Once trained, the same embedding serves the three tasks of face recognition named in the paper’s title and introduction: verification by thresholding a distance, identification by nearest neighbours, and clustering. The publication, with its private training set of 100 to 200 million images, its ablations, its LFW and YouTube Faces results and the differences between its versions, is the subject of the FaceNet paper article; this article covers FaceNet as a model as it is understood and used today: the loss and mining as later implementations read them, the networks, the open reimplementations and their trained models, and how margin-based softmax losses took over its training role.

Problem Addressed

Deep face recognizers of 2014, such as DeepFace and the DeepID series, trained a network to classify thousands of training identities and then reused an intermediate layer as the face descriptor, usually with further steps such as PCA, a Joint Bayesian model or an SVM on top (Schroff et al., 2015). FaceNet’s authors objected that this is indirect: a layer trained to separate the training identities is only hoped to separate new people, and it is high-dimensional. Their alternative was to train the network end to end so that its normalized output is the representation, with the loss defined on the distances that the system will threshold and rank at test time. This is the framing of metric learning: the people a deployed system must recognize are not among the training classes, so what is learned is a distance, not a classifier.

Architecture

FaceNet is a recipe more than a fixed network: any deep CNN followed by a fully connected layer to $d = 128$ outputs and $\ell_2$ normalization,

$$f(x) = \frac{g(x)}{\lVert g(x) \rVert_2} \in \mathbb{R}^{128}, \qquad \lVert f(x) \rVert_2 = 1,$$

where $g$ is the network’s raw output for a face crop $x$. All embeddings lie on the unit hypersphere, so the squared distance between two of them lies between 0 and 4 and equals $2 - 2\cos\theta$, with $\theta$ the angle between them (similarity and distance measures). The paper used 128 dimensions in all its main experiments and reports that the float vector can be quantized to 128 bytes per face without loss of accuracy; its comparison of dimensionalities is in the paper article.

Networks in the paper

The paper compares two families of networks under the same loss (Schroff et al., 2015, Tables 1–3):

• NN1, a network after Zeiler and Fergus (Zeiler–Fergus paper) with $1 \times 1$ convolutions inserted between its convolutional layers and two fully connected maxout layers: 22 layers deep, $220 \times 220$ inputs, 140 million parameters and 1.6 billion multiply-adds (“FLOPS” in the paper), most of the parameters in the fully connected layers.
• NN2 to NNS2, networks based on the Inception design of GoogLeNet. NN2 is close to GoogLeNet, with $\ell_2$ pooling in some modules, an average pool, a fully connected layer to 128 dimensions and the normalization: 7.5 million parameters and 1.6 billion multiply-adds on $224 \times 224$ inputs. NN3 and NN4 take $160 \times 160$ and $96 \times 96$ inputs, and the “mini” and “tiny” NNS1 and NNS2 were sized for phones, NNS2 running at 30 ms per image on a mobile phone.

By the printed sizes, NN2 has about a twentieth of NN1’s parameters at the same 1.6 billion FLOPS. How the networks compare in accuracy on the paper’s private test sets, and the inconsistencies between its Table 3 and Figure 4, are set out in the paper article.

Backbones in later implementations

The recipe does not depend on the backbone, and open reimplementations changed it. OpenFace kept FaceNet’s NN4 design in a reduced form, nn4.small2, on $96 \times 96$ inputs aligned by an affine transform from facial landmarks (Amos, Ludwiczuk & Satyanarayanan, 2016). David Sandberg’s TensorFlow repository defaults to Inception-ResNet-v1 on $160 \times 160$ crops (Sandberg, src/train_tripletloss.py lines 432–439 and src/train_softmax.py lines 494–501 at commit 096ed77), a hybrid of Inception modules and residual connections that Szegedy et al. (2017) introduced with roughly the computational cost of Inception-v3. The PyTorch port of Sandberg’s models implements the same network with a 512-dimensional output followed by batch normalization and $\ell_2$ normalization (Esler, models/inception_resnet_v1.py at commit 787da06). The ResNet backbones of the margin-loss era are described in ArcFace.

Training

The triplet loss in use

For a triplet $(x^a, x^p, x^n)$ the loss is the hinge

$$\ell = \Big[\, \lVert f(x^a) - f(x^p) \rVert_2^2 - \lVert f(x^a) - f(x^n) \rVert_2^2 + \alpha \,\Big]_+ ,$$

summed over the selected triplets in the paper and averaged in Sandberg’s triplet_loss function (src/facenet.py, lines 44–62). Three choices define FaceNet’s version among the variants compared in the triplet loss article: the distance is squared, the embeddings are unit length, and the margin is $\alpha = 0.2$, which on unit vectors asks for a cosine gap of 0.1 between the anchor’s positive and its negative. Both open reimplementations keep these values: OpenFace uses FaceNet’s loss on the unit hypersphere with $\alpha = 0.2$ and 128 dimensions “as suggested in the FaceNet paper” (Amos et al., 2016, Appendix A), and Sandberg’s triplet script defaults to $\alpha = 0.2$ and a 128-dimensional embedding (lines 446–449), normalized at line 136. The squared distance has a consequence that the triplet loss article derives: the gradient that pushes a negative away is proportional to its distance from the anchor, so it vanishes for the hardest negatives, the ones lying almost on top of the anchor. Wu et al. (2017) pointed to this as the reason hard-negative mining with the squared loss collapses, all embeddings gathering at one point.

The paper trained with stochastic gradient descent and AdaGrad, a starting learning rate of 0.05, from random initialization, on a CPU cluster for 1,000 to 2,000 hours, and observed that the loss decreased much more slowly after about 500 hours while accuracy kept improving.

Online semi-hard mining

The ideal training signal for each anchor would be its farthest positive and its closest negative over the whole training set. The paper rejects this as infeasible, and as likely to be dominated by mislabelled and poorly imaged faces, and selects triplets online, within each mini-batch (Schroff et al., 2015, §3.2):

1. Batches with many images per identity. About 40 faces of each identity per batch, plus randomly sampled negative faces, in batches of about 1,800 images, a size the authors say was set by the need to find hard triplets inside the batch rather than by optimization.
2. All positives. Every anchor–positive pair in the batch is used instead of only the hardest positive, which the authors found more stable and slightly faster to converge early in training.
3. Semi-hard negatives. Taking the hardest negative “can in practice lead to bad local minima early on in training” (Schroff et al., 2015), including a collapsed model. Negatives are instead chosen with $\lVert f(x^a) - f(x^p) \rVert_2^2 < \lVert f(x^a) - f(x^n) \rVert_2^2$ (the paper’s Eq. 3 in the CVPR text), farther from the anchor than the positive, which the text adds lie inside the margin.

The rule as printed leaves two things open: whether a semi-hard negative must also lie inside the margin, which Eq. 3 does not require but the text says, and which of several qualifying negatives to take. Implementations filled the gap differently:

Reading	Condition on $d_{an}$ (squared distances)	Choice among qualifying negatives	Source
closest beyond the positive	$d_{an} > d_{ap}$	the closest	Wu et al. (2017), formalizing FaceNet
semi-hard band	$d_{ap} < d_{an} < d_{ap} + \alpha$	uniformly at random	Sandberg’s code, line 295, commented “FaceNet selection” (disabled)
any margin violator	$d_{an} < d_{ap} + \alpha$	uniformly at random	Sandberg’s code, line 296 (active), commented “VGG Face” selection
within the margin (OpenFace)	a negative within the margin; pairs without one are not used	not stated	Amos et al. (2016), §3.2

In Sandberg’s select_triplets function (src/train_tripletloss.py at commit 096ed77, lines 271–311), a comment headed “VGG Face” (lines 279–284) explains the active rule: a random negative among those that violate the margin is a milder and cheaper form of hard-negative mining than the maximally violating one. The README credits ideas from the “Deep Face Recognition” paper of Oxford’s Visual Geometry Group, to which, in our reading, the comment’s heading refers. The line implementing FaceNet’s band is present but commented out, so the repository’s triplet training includes hard negatives, drawn at random; a pair with no qualifying negative yields no triplet (line 298). Its defaults are 45 people per batch with 40 images each (lines 440–443) and AdaGrad (lines 459–460). OpenFace sampled at most 20 images of each of 15 people per batch, used all anchor–positive pairs, and skipped a pair when no negative was found within the margin. Wu et al. (2017) describe semi-hard mining as widely adopted for the triplet loss and argue that it finds only a narrow band of negatives, which eventually runs out, so that training stops progressing; they read FaceNet’s report that the decrease of the loss slows drastically after some point as consistent with this. The general practice of training on hard examples is the subject of hard negative mining.

Example: selection rules on synthetic faces

The script below trains FaceNet’s configuration (unit-length embedding, squared distance, $\alpha = 0.2$, every ordered anchor–positive pair in the batch) on synthetic faces and changes only the rule that picks the negative. Each identity is a vector of eight geometric parameters (head size, eye spacing, height and size, nose length, mouth width and height) and each image a $32 \times 32$ rendering under random rotation, shift, scale, expression, lighting and noise, the renderer of the examples in face recognition and ArcFace; the small network is the ArcFace example’s, with a 64-dimensional output. Training uses 200 identities for 120 steps of Adam; each step renders fresh images of 16 random identities, 10 each, a scaled-down version of FaceNet’s batches of many images per identity. The four triplet rules are the hardest negative, which the paper warns against, and the first three readings of the table above; pairs with no qualifying negative are skipped, as in Sandberg’s code. A fifth run trains the same network on the same image budget with a margin-based softmax of the CosFace form ($s = 16$, $m = 0.35$, the values of the ArcFace example), one weight vector per training identity and no mining.

Over the last 30 steps the script records the share of anchor–positive pairs that found a qualifying negative (“kept”), the share of the selected triplets with positive loss (“active”), the mean squared distance between embeddings in the batch (0 means collapse; independent random unit vectors would give 2 on average, by our arithmetic) and the share of all valid triplets in the batch that violate the margin. It then embeds 10 images each of 150 identities never seen in training and scores the three uses: verification as the true accept rate at a false accept rate of $10^{-3}$ over all pairs (the threshold is set from the impostor scores, as in the ROC curve), closed-set identification as rank-1 accuracy with the first image of each identity as the gallery, and clustering by average-linkage agglomerative clustering on cosine distance with the true number of clusters, scored by the adjusted Rand index (1 for a perfect partition, about 0 for a random one). Choosing a gallery identity at random would give a rank-1 accuracy of $1/150 \approx 0.007$. The script uses PyTorch 2.14.1 and scikit-learn 1.9.1 on four CPU threads and took 100 to 150 seconds in our runs; two runs printed identical output. It is shown as first run, with no changes afterwards.

import torch
import torch.nn as nn
import torch.nn.functional as F
from sklearn.cluster import AgglomerativeClustering
from sklearn.metrics import adjusted_rand_score

torch.set_num_threads(4)
H = 32
yy, xx = torch.meshgrid(torch.arange(H) - 15.5, torch.arange(H) - 15.5, indexing="ij")

def render(p, gen):
    """One 32x32 face per row of p (8 identity parameters in [0, 1]): ellipse head, eyes,
    nose, mouth, under random rotation, shift, scale, expression, lighting and noise."""
    n = len(p)
    U = lambda lo, hi, *s: lo + (hi - lo) * torch.rand(n, *s, generator=gen)
    L = lambda k, lo, hi: (lo + (hi - lo) * p[:, k])[:, None, None]
    th, sc = torch.deg2rad(U(-8, 8))[:, None, None], U(0.94, 1.06)[:, None, None]
    sx, sy = U(-1.5, 1.5)[:, None, None], U(-1.5, 1.5)[:, None, None]
    x = ((xx - sx) * th.cos() + (yy - sy) * th.sin()) / sc
    y = (-(xx - sx) * th.sin() + (yy - sy) * th.cos()) / sc
    a, b = L(0, 8, 13), L(1, 10.5, 15)                          # head half-axes
    ex, ey, er = L(2, 3, 7), L(3, -7, -2), L(4, 1, 2.6)         # eye spacing, height, size
    nl, mw, my = L(5, 1, 5.5), L(6, 2.5, 7.5), L(7, 3.5, 8.5)   # nose, mouth width, height
    mw = mw * U(0.85, 1.15)[:, None, None]                      # expression: smile width
    lid = U(0.5, 1.0)[:, None, None]                            # expression: eye opening
    head = (x / a) ** 2 + (y / b) ** 2 < 1
    eyes = ((x.abs() - ex) / er) ** 2 + ((y - ey) / (er * lid)) ** 2 < 1
    nose = (x.abs() < 0.8) & (y > ey + 1) & (y < ey + 1 + nl)
    mouth = (x.abs() < mw) & ((y - my).abs() < 1.0)
    g = U(-0.03, 0.03, 2)[:, :, None, None]
    img = (U(0.1, 0.4)[:, None, None] + head * U(0.4, 0.6)[:, None, None]
           - (eyes | nose | mouth) * head * U(0.35, 0.5)[:, None, None]
           + g[:, 0] * xx + g[:, 1] * yy) * U(0.7, 1.3)[:, None, None]
    img = img + 0.06 * torch.randn(n, H, H, generator=gen)
    img = (img - img.mean((1, 2), keepdim=True)) / img.std((1, 2), keepdim=True)
    return img[:, None]

def embedder(d=64):
    return nn.Sequential(
        nn.Conv2d(1, 16, 3, stride=2, padding=1), nn.BatchNorm2d(16), nn.ReLU(),
        nn.Conv2d(16, 32, 3, padding=1), nn.BatchNorm2d(32), nn.ReLU(), nn.MaxPool2d(2),
        nn.Conv2d(32, 64, 3, padding=1), nn.BatchNorm2d(64), nn.ReLU(), nn.MaxPool2d(2),
        nn.Flatten(), nn.Linear(64 * 4 * 4, d))

def select(D, y, rule, alpha, gen):
    """FaceNet-style online selection: every (anchor, positive) pair in the batch,
    one negative per pair chosen by `rule`; pairs without a qualifying negative are skipped."""
    same = y[:, None] == y[None]
    a, p = (same & ~torch.eye(len(y), dtype=torch.bool)).nonzero(as_tuple=True)
    dap, dan = D[a, p][:, None], D[a]                     # squared distances
    neg = ~same[a]
    if rule == "hardest":
        ok = neg
    elif rule == "semi-hard closest":                     # Eq. 3, closest qualifying negative
        ok = neg & (dan > dap)
    elif rule == "semi-hard band":                        # d_ap < d_an < d_ap + alpha, random
        ok = neg & (dan > dap) & (dan < dap + alpha)
    elif rule == "violating random":                      # d_an < d_ap + alpha, random
        ok = neg & (dan < dap + alpha)
    keep = ok.any(1)
    if rule in ("hardest", "semi-hard closest"):
        n = dan.masked_fill(~ok, float("inf")).argmin(1)
    else:
        n = torch.multinomial(ok.float() + 1e-12 * ~keep[:, None], 1, generator=gen).squeeze(1)
    return a[keep], p[keep], n[keep], keep.float().mean().item()

def train(P, rule, gen, steps=120, n_id=16, per_id=10, alpha=0.2, s=16.0, m=0.35):
    f = embedder()
    W = nn.Parameter(0.1 * torch.randn(len(P), 64, generator=gen))     # used only by CosFace
    opt = torch.optim.Adam(list(f.parameters()) + [W], lr=3e-3)
    log = []
    for step in range(steps):
        if rule == "CosFace":                             # 160 random images, no mining
            y = torch.randint(len(P), (n_id * per_id,), generator=gen)
        else:                                             # 16 identities x 10 images
            y = torch.randperm(len(P), generator=gen)[:n_id].repeat_interleave(per_id)
        z = F.normalize(f(render(P[y], gen)))             # unit-length embeddings
        D = torch.cdist(z, z) ** 2                        # squared Euclidean distances
        if rule == "CosFace":
            cos = z @ F.normalize(W).T
            loss = F.cross_entropy(s * (cos - m * F.one_hot(y, len(P))), y)
            kept = act = float("nan")
        else:
            a, p, n, kept = select(D.detach(), y, rule, alpha, gen)
            terms = (D[a, p] - D[a, n] + alpha).clamp(min=0)
            loss = terms.mean() if len(terms) else D.sum() * 0
            act = (terms > 0).float().mean().item() if len(terms) else 0.0
        opt.zero_grad(); loss.backward(); opt.step()
        if step >= steps - 30:                            # diagnostics, last 30 steps
            Dd, same = D.detach(), y[:, None] == y[None]
            valid = (same & ~torch.eye(len(y), dtype=torch.bool))[:, :, None] & ~same[:, None, :]
            viol = ((Dd[:, :, None] - Dd[:, None, :] + alpha > 0) & valid).sum() / valid.sum()
            log.append([kept, act, Dd[~torch.eye(len(y), dtype=torch.bool)].mean().item(), viol.item()])
    return f.eval(), torch.tensor(log).mean(0).tolist()

def uses(E, ids):
    """Verification (TAR at FAR 1e-3), identification (rank-1) and clustering (ARI)."""
    S = E @ E.T
    iu = torch.triu_indices(len(ids), len(ids), 1)
    sim, same = S[iu[0], iu[1]], ids[iu[0]] == ids[iu[1]]
    imp = sim[~same]
    t = imp.topk(int(1e-3 * len(imp)) + 1).values[-1]     # threshold admitting 1 impostor in 1,000
    tar = (sim[same] > t).float().mean().item()
    first = torch.ones(len(ids), dtype=torch.bool); first[1:] = ids[1:] != ids[:-1]
    r1 = (ids[first][S[~first][:, first].argmax(1)] == ids[~first]).float().mean().item()
    lab = AgglomerativeClustering(n_clusters=int(ids.max()) + 1, metric="cosine",
                                  linkage="average").fit_predict(E.numpy())
    return tar, r1, adjusted_rand_score(ids.numpy(), lab)

print("PyTorch", torch.__version__)
print("seed  rule               kept  active  mean d2  viol(all)   TAR@1e-3  rank-1    ARI")
rules = ["hardest", "semi-hard closest", "semi-hard band", "violating random", "CosFace"]
for seed in range(3):
    for rule in rules:
        gen = torch.Generator().manual_seed(seed); torch.manual_seed(seed)
        P_train, P_test = torch.rand(200, 8, generator=gen), torch.rand(150, 8, generator=gen)
        f, (kept, act, d2, viol) = train(P_train, rule, gen)
        ids = torch.arange(150).repeat_interleave(10)     # 150 unseen identities x 10 images
        with torch.no_grad():
            E = F.normalize(f(render(P_test[ids], gen)))
        tar, r1, ari = uses(E, ids)
        print(f"{seed:4d}  {rule:17s} {kept:5.2f}  {act:6.2f}  {d2:7.3f}  {viol:9.3f}"
              f"   {tar:8.3f}  {r1:6.3f}  {ari:5.3f}")

PyTorch 2.14.1
seed  rule               kept  active  mean d2  viol(all)   TAR@1e-3  rank-1    ARI
   0  hardest            1.00    1.00    0.000      1.000      0.012   0.061  0.020
   0  semi-hard closest  1.00    0.36    1.750      0.012      0.507   0.833  0.688
   0  semi-hard band     0.75    1.00    1.573      0.059      0.210   0.580  0.373
   0  violating random   0.40    1.00    1.672      0.015      0.491   0.761  0.630
   0  CosFace             nan     nan    0.907      0.251      0.156   0.407  0.262
   1  hardest            1.00    1.00    0.000      1.000      0.012   0.070  0.024
   1  semi-hard closest  1.00    0.39    1.701      0.015      0.518   0.810  0.674
   1  semi-hard band     0.59    1.00    1.651      0.031      0.356   0.686  0.549
   1  violating random   0.36    1.00    1.737      0.013      0.483   0.797  0.661
   1  CosFace             nan     nan    0.871      0.257      0.124   0.356  0.223
   2  hardest            1.00    1.00    0.000      1.000      0.012   0.067  0.026
   2  semi-hard closest  1.00    0.38    1.722      0.013      0.534   0.820  0.696
   2  semi-hard band     0.50    1.00    1.771      0.022      0.432   0.747  0.613
   2  violating random   0.41    1.00    1.743      0.015      0.551   0.819  0.699
   2  CosFace             nan     nan    0.826      0.281      0.096   0.270  0.182

Hardest negatives collapsed. In all three seeds the mean squared distance in the batch fell to 0.000 and every triplet in the batch violated the margin: all faces were mapped to the same point of the sphere, and on new identities the embedding scores far below the other rules, if above chance (rank-1 of 0.061 to 0.070 against 0.007). This is the collapse FaceNet describes; with normalized outputs it cannot take the form $f(x) = 0$ that the paper names, but a single point serves equally. The vanishing push on hard negatives under the squared distance, described above, is consistent with it, although the run does not isolate that cause.

Every semi-hard reading trained. The other three rules learned embeddings that verify, identify and cluster new identities far above chance, and the way they spent the batch differed as their definitions suggest. The closest-beyond-the-positive rule kept nearly every pair, since some negative was almost always farther than the positive, but only 36% to 39% of its triplets had positive loss: the rest lay beyond the margin and contributed nothing. The rule that draws any margin violator at random found a negative for only 36% to 41% of pairs by the end of training, because the rest were already satisfied, and every triplet it chose was active. The two ended close together, with the closest rule ahead in TAR in seeds 0 and 1 and the random violator in seed 2. Drawing at random among all violators includes hard negatives but did not collapse; in our reading, a random draw rarely lands on the very closest negative, so few of its triplets are of the kind that the hardest rule chooses every time.

The band alone was weakest. Random negatives restricted to FaceNet’s band, with no hard negatives at all, gave the lowest scores of the three rules that trained in every seed (TAR 0.210 to 0.432) and left more of the batch’s triplets violated. Our reading, which the run does not test directly, is that this rule never selects a negative that is already closer than the positive, so such violations are corrected only indirectly. The spread across seeds is also largest for this rule.

The margin softmax did not catch up at this budget. Against the expectation recorded before the run, that it would reach a TAR “at least as high as the best triplet rule at equal image budget”, the CosFace-form loss gave a TAR of 0.096 to 0.156, below every triplet rule that trained, with its embeddings less spread out (mean squared distance 0.83 to 0.91). The settings were fixed in advance and not tuned: the same optimizer, learning rate and number of images as the triplet runs. The result shows that a margin loss is not automatically better in a small, short run; it says nothing about the comparisons at scale in the next section, where the same network and training set were used for both and the margin losses won, and its numbers are not comparable with those of the ArcFace article’s example, which uses a different schedule.

Inference

At test time FaceNet is a function from a face crop to a vector; everything else is distance computation:

• Input. A face located by a face detector. The paper used a tight crop of the detected box, scaled to the network’s input size, with no further alignment. Reimplementations align the face first: OpenFace by an affine transform that moves the eye corners and nose to fixed positions, Sandberg’s models by crops from MTCNN, with “fixed image standardization” of the pixel values for the 2018 models (Sandberg, README).
• Verification. Two faces are judged to be the same person when the squared distance between their embeddings is below a threshold. On LFW the paper selected the threshold on nine folds and applied it to the tenth (see the paper article). The threshold sets the trade-off between false accepts and false rejects and is chosen for the application’s operating point, as described in face recognition.
• Identification. A probe is compared with a gallery of enrolled embeddings and assigned to the nearest identity, or rejected when even the nearest is beyond the threshold; the paper describes recognition as k-nearest-neighbour classification.
• Clustering. The embeddings can be grouped by off-the-shelf algorithms such as k-means or agglomerative clustering; the paper shows one cluster from a personal photo collection obtained by agglomerative clustering, without a quantitative evaluation.

Because the embedding is computed once per face, new people need no retraining: enrolling a person means storing a vector. For collections of many faces, the comparison becomes nearest-neighbour search over vectors (image retrieval).

Performance

The published results, 99.63% on LFW under the protocol for unrestricted, labeled outside data and 95.12% on YouTube Faces, came from Google’s private training data and are discussed with their protocol in the paper article and in Labeled Faces in the Wild. The paper reports nothing about the demographic composition of its training or test data and no breakdown of errors by group.

Open reimplementations trained on public data reported LFW accuracies from 0.9292 to 0.9965, each under its own pipeline:

Model	Training data	Loss	LFW accuracy
OpenFace nn4.small2 (2016)	about 500,000 images of CASIA-WebFace and FaceScrub	triplet	0.9292 ± 0.0134
Sandberg 20180408-102900	CASIA-WebFace	softmax	0.9905
Sandberg 20180402-114759	VGGFace2	softmax	0.9965 (0.99650 ± 0.00252)

The OpenFace figure is from its technical report and the Sandberg figures from the repository README, which points to a wiki page, not read for this article, for how the test was run. Neither source reports demographic composition. These numbers are not comparable with each other or with the paper’s: the training data, alignment and network differ in every row.

Triplet training against margin losses

Comparisons with everything else fixed favour the margin-based softmax losses. SphereFace’s authors trained the same 64-layer network on CASIA-WebFace with several losses (Liu et al., 2017, Tables 4 and 5):

Loss (same network, CASIA-WebFace)	LFW	YouTube Faces	MegaFace rank-1, 1M distractors	MegaFace TAR at FAR $10^{-6}$
softmax	97.88	93.1	54.855	65.925
triplet loss	98.70	93.4	64.797	78.322
SphereFace (A-Softmax)	99.42	95.0	72.729	85.561

All values are percentages, for a single model; the MegaFace columns are for its protocol with a small training set. In the ArcFace paper’s ablation on CASIA-WebFace with a ResNet50 (Deng et al., 2019, Table 2), an angular version of the triplet loss with margin 0.35 reached 98.98% on LFW, 91.90% on CFP-FP and 89.98% on AgeDB-30, against 99.53%, 95.56% and 95.15% for ArcFace with $m = 0.5$; adding the triplet loss to ArcFace gave 99.50%, 95.51% and 94.40%, which the authors read as no improvement. In the same table the plain softmax also beat the triplet loss on all three benchmarks (99.08%, 94.39% and 92.33%), the reverse of SphereFace’s comparison, so the order of those two depends on the setup.

Strengths

• One compact representation for three tasks. Verification, identification and clustering all reduce to distances between 128-dimensional unit vectors, and new identities require no retraining.
• Training on the test-time quantity. The loss is defined on the distances that are thresholded and ranked at test time, with no classifier to discard and no post-processing such as PCA or a learned metric on top.
• No weights per training identity. Unlike classification losses, whose final layer grows linearly with the number of identities (Deng et al., 2019), the triplet loss has no per-identity parameters, so, by our reasoning, its model size does not grow with the number of training identities.
• Backbone-independent. The same loss and mining were used with Zeiler–Fergus, Inception and Inception-ResNet networks, and with networks small enough for phones.

Limitations

• Mining is the hard part. Of the $O(n^3)$ triplets in a training set, few are informative after early training, and which ones are chosen decides whether training learns, stalls or collapses. ArcFace’s authors name the combinatorial explosion of triplets and the difficulty of semi-hard mining as the triplet loss’s two drawbacks, and SphereFace’s call pair and triplet mining “both time-consuming and performance-sensitive” (Liu et al., 2017).
• Large batches and long training. The paper’s mining needed batches of about 1,800 images with 40 per identity, and 1,000 to 2,000 CPU-cluster hours per model, with progress slowing after about 500 hours.
• An underspecified rule. The semi-hard condition and the choice among qualifying negatives are not fully specified, and the readings in use differ; the example above shows that the reading changes the result.
• Relative constraints. Each triplet compares two distances from one anchor, so a model can satisfy every triplet while identities differ in spread; verification needs a single global threshold, which the loss does not directly optimize (see the triplet loss article).
• Data. The original model was trained on private data whose composition the paper does not describe. Public reimplementations were trained on web-collected datasets, CASIA-WebFace and VGGFace2, whose provenance and later status are described in face recognition; the measured demographic differentials of face recognition algorithms, summarized there from NIST’s reports, apply to embeddings of this kind as to any other.

Historical Importance

FaceNet showed in 2015 that a face recognizer could be trained end to end as an embedding for comparison, with no classification stage, and the paper presented its LFW result as a new record (see the paper article). In our reading, several of its choices outlived its loss: a compact, $\ell_2$-normalized face embedding; comparison by distance or cosine; and verification, identification and clustering as uses of one vector. Its online semi-hard mining became, in Wu et al.’s words, widely adopted for the triplet loss, and later work on sampling in deep metric learning started from it (Wu et al., 2017); those strategies are compared in the triplet loss article.

Its training objective was replaced within a few years. From 2017 the field returned to classification over identities, but with normalized class weights (and, in CosFace and ArcFace, normalized features as well) and an angular or cosine margin: SphereFace (2017), CosFace (2018) and ArcFace (2019), covered in ArcFace. These losses compare each image with every class weight at every step, so they need no mining, and in the controlled comparisons above they beat the triplet loss; their cost is a weight vector per training identity. The open reimplementations show the same shift in practice: Sandberg’s repository, which describes itself as an implementation of FaceNet, states in its README that the best results came from training with a softmax loss, and its released models were trained that way. What survived from FaceNet is the test-time model: a normalized embedding of a face, compared by distance with people never seen in training.

Variants

• Network sizes. The paper’s NN1 to NNS2 trade accuracy for computation, from 1.6 billion multiply-adds down to the “tiny” NNS2 for phones (see Architecture).
• Harmonic embeddings. An appendix of the later arXiv versions trains a new network whose embeddings remain comparable with an older network’s, so that stored embeddings need not be recomputed after an upgrade; it is described in the paper article.
• Distilled networks. The MobileNets paper trained small networks to reproduce FaceNet’s outputs, as described there.
• Reimplementations. OpenFace’s nn4.small2 kept the triplet loss on public data; Sandberg’s Inception-ResNet-v1 models kept the name and the test-time embedding but were trained with a softmax loss (see Implementations).

Implementations

No official code or trained model accompanied the paper. As of November 2026:

• davidsandberg/facenet (TensorFlow, MIT licence, copyright 2016 David Sandberg). The README describes it as a TensorFlow implementation of the FaceNet paper, heavily inspired by OpenFace, tested with TensorFlow r1.7; the last commit is from 16 April 2018. It contains a triplet-loss training script (train_tripletloss.py) with the selection rule described above, a softmax training script with an optional center loss, MTCNN face alignment, and two released models, 20180408-102900 (CASIA-WebFace, 453,453 images of 10,575 identities after face detection, by the README’s count) and 20180402-114759 (VGGFace2, about 3.3 million faces of about 9,000 classes), both Inception-ResNet-v1 trained with softmax loss. The README asks users of the models to credit the providers of the training data and states no separate licence for the trained weights.
• timesler/facenet-pytorch (PyTorch, MIT licence, copyright 2019 Timothy Esler; last commit 2 August 2024). It ports Sandberg’s two models to PyTorch, initialized from his parameters, with the LFW accuracies listed in his README, and returns 512-dimensional embeddings by default, or class logits over the training identities (8,631 for VGGFace2, 10,575 for CASIA-WebFace). It also includes an MTCNN implementation for detection and alignment.
• OpenFace (Torch and Python, Apache 2.0 licence; last commit 18 July 2026), released in October 2015 by Carnegie Mellon University: the nn4.small2 network trained with FaceNet’s triplet loss on CASIA-WebFace and FaceScrub (Amos et al., 2016).

The margin-loss successors are implemented in the InsightFace code described in ArcFace. Triplet losses and miners for training a FaceNet-style embedding in PyTorch are compared in the triplet loss article.