ArcFace

ArcFace is a loss function for training face recognition networks: a softmax classifier over the training identities whose features and class weights are normalized, so that every logit is a cosine, and whose target logit is penalized by adding a fixed angle $m$ to the angle between the feature and its class weight. It belongs to a family of margin-based softmax losses that also includes the normalized softmax (NormFace), the multiplicative angular margin of L-Softmax and SphereFace, and the additive cosine margin of CosFace and AM-Softmax. All of them train an embedding by classification and then discard the classifier: at test time faces of people never seen in training are compared by cosine similarity, as described in face recognition. This article covers the family, from the plain normalized softmax to ArcFace and its later variants, the geometry of each margin on the hypersphere, how implementations compute the ArcFace logit, what goes wrong in training, and the results the papers report. Pair, triplet and proxy losses are covered in metric learning and triplet loss.

Definition

A margin-based softmax loss is the cross-entropy loss of a linear classifier without bias in which the logit of the true class $y_i$ is replaced by a smaller value. In all but the earliest members, the feature $x_i$ and each class weight $W_j$ are $\ell_2$-normalized, so that the logit of class $j$ is $s \cos\theta_j$ with $\theta_j$ the angle between $x_i$ and $W_j$ and $s$ a fixed or learned scale, and the true class gets $s\,\psi(\theta_{y_i})$ instead; L-Softmax normalizes neither vector and SphereFace only the weights. The margin function $\psi$ makes classification harder during training than at the decision boundary of the plain classifier: a feature counts as correctly classified only if it is closer to its own class weight than to any other by a margin. The members of the family differ in where the margin acts: on the angle multiplicatively (SphereFace, $\cos m\theta$), on the cosine additively (CosFace, $\cos\theta - m$), or on the angle additively (ArcFace, $\cos(\theta + m)$).

Intuition

A softmax classifier trained on identities learns features that separate the training classes, but separation is all it asks for: a feature just on the right side of a decision boundary has as low a loss as one deep inside its class, and the boundary between two similar people can pass close to both. At test time the classes are different people, and verification thresholds a similarity between two embeddings, so what matters is that images of one person form a tight cluster with empty space around it. The margin losses ask for that space directly.

Normalization turns the problem into one about directions. With unit-length features and class weights, each class weight is a point on the unit hypersphere, a kind of learned class centre, and classification depends only on angles. NormFace calls the weight vector an “agent” of its class (Wang et al., 2017), and the metric learning literature calls it a proxy. A margin then means: the angle from a feature to its own centre must be smaller than the angle to every other centre by some amount. Because the classifier compares each feature with every class centre at every step, no pairs or triplets have to be sampled, which was the motivation the SphereFace and ArcFace papers gave for preferring it to the triplet loss (Liu et al., 2017; Deng et al., 2019).

Formal Definition

Softmax and its normalized form

For a feature $x_i \in \mathbb{R}^d$ with label $y_i$ among $C$ training identities, the softmax cross-entropy with weights $W_j$ and biases $b_j$ is

$$L_i = -\log \frac{e^{W_{y_i}^\top x_i + b_{y_i}}}{\sum_{j=1}^{C} e^{W_j^\top x_i + b_j}}, \qquad W_j^\top x_i = \lVert W_j \rVert\, \lVert x_i \rVert \cos\theta_j .$$

Setting $b_j = 0$, $\lVert W_j \rVert = 1$ and $\lVert x_i \rVert = s$ gives the normalized softmax, whose logits are $s\cos\theta_j$. L-Softmax already left out the bias (Liu et al., 2016); SphereFace also normalized the weights, so that the binary decision boundary becomes the angular bisector $\cos\theta_1 = \cos\theta_2$ (Liu et al., 2017). Normalizing the feature as well, and multiplying the cosine by a scale, came from Ranjan, Castillo and Chellappa (2017), who constrained features to a hypersphere of radius $\alpha$ (the L2-softmax), and from NormFace (Wang et al., 2017), which normalized both features and weights and appended a learnable scale.

The margin losses

All the losses below have the form

$$L_i = -\log \frac{e^{s\,\psi(\theta_{y_i})}}{e^{s\,\psi(\theta_{y_i})} + \sum_{j \neq y_i} e^{s \cos\theta_j}},$$

with the non-target logits left as $s\cos\theta_j$:

Loss	Target logit $\psi(\theta)$	Normalized	Typical setting
Normalized softmax (NormFace)	$\cos\theta$	$W$ and $x$, scale $s$	$s$ learned, or fixed at 20 to 30
L-Softmax (Liu et al., 2016)	$\lVert W \rVert \lVert x \rVert\, \psi_m(\theta)$	neither	$m = 4$
SphereFace, A-Softmax (Liu et al., 2017)	$\lVert x \rVert\, \psi_m(\theta)$	$W$ only	$m = 4$, annealed (below)
CosFace (Wang et al., 2018a), AM-Softmax (Wang et al., 2018b)	$\cos\theta - m$	$W$ and $x$	$m = 0.35$; $s = 64$ (CosFace), $s = 30$ (AM-Softmax)
ArcFace (Deng et al., 2019)	$\cos(\theta + m)$	$W$ and $x$	$m = 0.5$, $s = 64$
Combined margin (Deng et al., 2019)	$\cos(m_1\theta + m_2) - m_3$	$W$ and $x$	e.g. $(1, 0.3, 0.2)$

Multiplicative angular margin. L-Softmax asks that a feature of class 1 satisfy $\cos(m\theta_1) > \cos\theta_2$ for an integer $m$, which is stricter than $\cos\theta_1 > \cos\theta_2$ because $\cos\theta \ge \cos m\theta$ for $\theta \in [0, \pi/m]$. Outside that interval $\cos m\theta$ is not monotonic, so both L-Softmax and SphereFace replace it by the piecewise function

$$\psi_m(\theta) = (-1)^k \cos(m\theta) - 2k, \qquad \theta \in \left[\tfrac{k\pi}{m}, \tfrac{(k+1)\pi}{m}\right],\ k = 0, \dots, m-1,$$

which equals $\cos m\theta$ on the first interval, decreases monotonically on $[0, \pi]$ and is continuous; with $m = 1$ it is the ordinary cosine (Liu et al., 2016, 2017). L-Softmax computes $\cos m\theta$ from $\cos\theta$ with the multiple-angle expansion, a polynomial in $W^\top x / (\lVert W \rVert \lVert x \rVert)$. For its ideal criterion, that the largest intra-class angle be smaller than the smallest inter-class angle, SphereFace derived lower bounds on $m$ of $2 + \sqrt{3}$ for two classes and 3 for many uniformly spaced classes, and used $m = 4$.

Additive cosine margin. CosFace, which its authors call the large margin cosine loss, and AM-Softmax, both first posted to arXiv in January 2018, subtract $m$ from the target cosine and normalize both vectors (Wang et al., 2018a, 2018b). AM-Softmax points out that the margin then needs no special gradient: $\partial(\cos\theta - m)/\partial\cos\theta = 1$.

Additive angular margin. ArcFace adds $m$ to the angle itself, computing $\theta_{y_i} = \arccos(W_{y_i}^\top x_i)$ for unit vectors and replacing the target logit by $s\cos(\theta_{y_i} + m)$ (Deng et al., 2019). Expanded, $\cos(\theta + m) = \cos\theta\cos m - \sin\theta \sin m$, so the penalty relative to the plain cosine depends on $\theta$ through $\sin\theta$; the first version of the paper calls the margin dynamic for this reason (Deng, Guo & Zafeiriou, 2018).

Combined margin. The CVPR paper puts the three margins into one target logit, $\cos(m_1\theta + m_2) - m_3$, with $(m_1, m_2, m_3) = (1, 0, 0)$ the normalized softmax, $(1, 0.5, 0)$ ArcFace and $(1, 0, 0.35)$ CosFace, and a version of SphereFace without the integer constraint, $(1.35, 0, 0)$, computed with the arc-cosine instead of SphereFace’s multiple-angle expansion (Deng et al., 2019).

Decision boundaries

In the binary case, with $\theta_1$ and $\theta_2$ the angles from a normalized feature to the two class weights, the boundaries for class 1 are (SphereFace Table 1; ArcFace arXiv v1, Table 1):

Loss	Boundary for class 1
Softmax	$(W_1 - W_2)^\top x + b_1 - b_2 = 0$
Normalized softmax	$\cos\theta_1 = \cos\theta_2$
SphereFace	$\cos m\theta_1 = \cos\theta_2$
CosFace	$\cos\theta_1 - m = \cos\theta_2$
ArcFace	$\cos(\theta_1 + m) = \cos\theta_2$

Class 2 has the mirrored boundary, and between the two lies a decision margin that no training feature should occupy. Along the arc between two class weights an angle $\varphi$ apart, the margin is the constant angle $m$ for ArcFace, which the paper calls a constant linear angular margin and links to the geodesic distance on the hypersphere. For SphereFace it is $\tfrac{m-1}{m+1}\varphi$, as the SphereFace paper derives, so it shrinks to nothing for classes whose weights are close, a weakness the CosFace paper points out. For CosFace it is, by our derivation from $\cos\theta_1 - \cos(\varphi - \theta_1) = m$, an angle of $2\arcsin\!\big(m / (2\sin\tfrac{\varphi}{2})\big)$, which grows as the weights get closer and exists only while $m \le 2\sin\tfrac{\varphi}{2}$. With weights $90°$ apart, CosFace’s $m = 0.35$ and ArcFace’s $m = 0.5$ rad give almost the same margin ($28.7°$ and $28.6°$), while SphereFace’s $m = 4$ gives $54°$; with weights $45°$ apart the three give $54.4°$, $28.6°$ and $27°$.

Properties

Why the scale is needed

A normalized logit lies in $[-1, 1]$, so the softmax probability of the true class cannot approach 1. NormFace makes this precise: if the $C$ classes have equal numbers of samples, each feature coincides with its class weight, and features and weights all have norm $\ell$, the loss is bounded below by

$$\log\!\left(1 + (C - 1)\, e^{-\frac{C}{C-1}\ell^2}\right).$$

With $\ell = 1$ on CASIA-WebFace ($C = 10{,}575$) the bound is 8.27, and the authors observed the training loss falling from about 9.27 only to about 8.50; multiplying the cosines by $s = \ell^2$ lowers the floor (Wang et al., 2017). Ranjan et al. (2017) gave a similar argument: requiring an average probability $p$ for $C$ classes placed $90°$ apart needs a radius of at least $\log\frac{p(C-2)}{1-p}$. CosFace derived a lower bound of the same form for the scale, in terms of the expected minimum posterior probability $P_W$ of a class centre,

$$s \ge \frac{C - 1}{C} \log \frac{(C-1)\,P_W}{1 - P_W}.$$

By our arithmetic this is about 11.5 for CASIA-WebFace at $P_W = 0.9$, so the common $s = 64$ is well above it. Larger is not always better: the TPAMI version of ArcFace reports that the normalized softmax with $s = 64$ lost accuracy on CFP-FP and AgeDB-30, and that a search found $s = 20$ best for it (Deng et al., 2022). AM-Softmax found that once a margin is added, a learned $s$ does not grow and training converges very slowly, so it fixed $s = 30$ (Wang et al., 2018b).

Target logit curves

What distinguishes the losses numerically is the curve $\psi(\theta)$ of the target logit against the angle. The CVPR paper plots these curves between $20°$ and $100°$, the range it observed during training: angles between features and their class weights start near $90°$ at random initialization and end near $30°$ (Deng et al., 2019, Figure 4). It names the start, the end and the slope of the curve as the factors that matter. CosFace shifts the curve down by $m$; ArcFace shifts it to the left by $m$. The first ArcFace version argued that a margin added at $60°$–$90°$ risks divergence, that a margin at $30°$–$60°$ improves accuracy because those are the semi-hard samples, and that one below $30°$ has little effect, calling $30°$ and $60°$ rough estimates (Deng, Guo & Zafeiriou, 2018).

The ArcFace curve is not monotonic: $\cos(\theta + m)$ is smaller than $\cos\theta$ only for $\theta \in [0, \pi - m]$, and beyond $\pi - m$ (about $151.35°$ for $m = 0.5$) it increases again. The first version noted this and argued that, since the initial angles are distributed around $90°$ with the largest below $105°$, the increasing part is almost never reached and needs no special treatment (Deng, Guo & Zafeiriou, 2018).

Gradients

How the margin acts in training is visible in the derivative of the target logit with respect to the cosine. For CosFace it is 1. For ArcFace computed with automatic differentiation through $\arccos$ it is, by the chain rule,

$$\frac{\partial \cos(\theta + m)}{\partial \cos\theta} = \frac{\sin(\theta + m)}{\sin\theta},$$

which is greater than 1 for $\theta < (\pi - m)/2$ (about $75.7°$), falls to zero at $\theta = \pi - m$ and is negative beyond it. A negative value means that gradient descent pushes such a feature away from its own class weight. This is our derivation; the first example below checks it numerically, and the third shows what it can do to training.

Class weights, norms and the number of classes

The class weights are not the class means. ArcFace’s angle statistics on CASIA-WebFace give a mean angle of $44.26°$ between each weight and the centre of its class’s training features under the normalized softmax, against $14.29°$ under ArcFace (Deng et al., 2019, Table 3). SphereFace’s appendix found that, without normalization, the norm of a class weight grows with the number of training samples of the class, and argued that normalizing the weights removes a prior learned from data imbalance that verification on new identities should not inherit (Liu et al., 2017, arXiv version). The norm of an unnormalized feature carries information too: Ranjan et al. (2017), citing earlier work, note that it is high for good-quality frontal faces and low for blurry faces in extreme poses, which MagFace and AdaFace later exploit (see Variants).

The classifier has one weight vector per identity, so its size grows linearly with the number of identities (Deng et al., 2019). As of October 2026, the official PyTorch implementation trains with a sparse variant, Partial FC, which samples a subset of class centres in each step; its README states that this keeps accuracy while training sets of up to 29 million identities (InsightFace, commit c6c3af7).

Implementation

The CVPR paper’s pseudo-code is nine MXNet steps: normalize features and weights, take the dot products, pick the target cosine, apply $\arccos$, add $m$, take the cosine, write the difference back into the target column through a one-hot mask, and multiply all logits by $s$ (Deng et al., 2019, Algorithm 1). Three ways of handling the non-monotonic part have been used in the official code:

• The formula as written. The combined-margin branch of the 2018 MXNet code computes $\cos(m_1\arccos(\cdot) + m_2) - m_3$ directly, with no guard (InsightFace, commit 22a0719, loss_type 5; the same code is in the current recognition/arcface_mxnet/train.py).
• A monotonic fallback. The default ArcFace branch of the same 2018 script (loss_type 4) computes $\cos\theta\cos m - \sin\theta\sin m$ while $\cos\theta > \cos(\pi - m)$, that is while $\theta < \pi - m$, and otherwise uses $\cos\theta - m\sin m$, a CosFace-like penalty. An “easy margin” option applies the margin only when $\cos\theta > 0$ and leaves the logit at $\cos\theta$ otherwise; it is off by default. The PyTorch code used the same rule from January 2022 (commit bb221e6) until December 2022; its first version, from March 2021, applied $\arccos$ and the cosine in place to logits that autograd then differentiated, so it computed the formula as written with no guard (commit ede87b4).
• Margin in the value, not in the gradient. Since December 2022 the PyTorch ArcFace and CombinedMarginLoss classes take $\arccos$, add $m$ and take the cosine in place inside torch.no_grad(), then multiply by $s$ (InsightFace, commits 657ae30 and c6c3af7). Autograd therefore sees the logit as $s\cos\theta$: the forward value is exactly $s\cos(\theta + m)$, but the derivative with respect to the cosine is $s$, as for CosFace, at every angle. The variables of the earlier rule are still defined in the constructor but no longer used.

As of October 2026 the default configuration, configs/base.py, sets margin_list = (1.0, 0.5, 0.0), as do the MS1MV2 and MS1MV3 configurations, while those for WebFace and Glint360K use CosFace, (1.0, 0.0, 0.4); train_v2.py fixes the scale at 64 and builds a CombinedMarginLoss that, beside ArcFace ($m_1 = 1$, $m_3 = 0$), supports only the CosFace case $m_3 > 0$, in which $m_1$ and $m_2$ are ignored, so a combined setting such as $(1, 0.3, 0.2)$ runs as CosFace with $m = 0.2$; any other setting raises an error.

Training tricks and failure modes

• SphereFace needed softmax supervision. L-Softmax was hard to optimize on data with many identities, so its authors trained with the target logit $\frac{\lambda \lVert W \rVert\lVert x \rVert\cos\theta + \lVert W \rVert\lVert x \rVert\psi(\theta)}{1 + \lambda}$, starting with a very large $\lambda$ (close to plain softmax) and reducing it (Liu et al., 2016). SphereFace’s official model file sets a base of 1000, a gamma of 0.12, a power of 1 and a minimum of 5, which its Caffe layer turns into $\lambda = \max(5,\ 1000/(1 + 0.12\,t))$ at iteration $t$ (SphereFace code, commit f20b130); this by our arithmetic reaches its floor of 5 after about 1,660 iterations. AM-Softmax describes the resulting curve with $m = 4$ and $\lambda = 5$ as approximately a margin of $m = 1.5$, and the CVPR ArcFace paper attributes SphereFace’s instability to the steep target logit of the integer margin. CosFace and AM-Softmax train from scratch with a fixed margin and no annealing.
• Too large a margin does not converge. CosFace reports that on CASIA-WebFace its model failed to converge for $m > 0.45$, and that accuracy on LFW and YouTube Faces saturated at $m = 0.35$ (Wang et al., 2018a). The first ArcFace version reports that SphereFace with $m = 2$ and $\lambda = 0$ did not converge (Deng, Guo & Zafeiriou, 2018).
• Small networks. CurricularFace reports that ArcFace with $m = 0.5$ on a MobileFaceNet trained on CASIA-WebFace did not converge, its loss becoming NaN at about the 2,400th step, while with $m = 0.45$ it converged (Huang et al., 2020).
• Label noise. The margin assumes clean labels. The TPAMI version reports that ArcFace’s TAR at FAR $= 10^{-4}$ on IJB-C with a ResNet50 fell from 96.50% when trained on MS1MV3, cleaned semi-automatically, to 90.27% when trained on its raw source MS1MV0, whose noise it puts at about 47.1% to 54.4% citing earlier work (Deng et al., 2022, Table 3). Sub-center ArcFace, below, was designed for this case.
• Feature normalization from scratch. CosFace argues that without normalized features its loss can be reduced by shrinking the feature norm whenever the margin cannot yet be met, so its model without normalization had to be initialized with a softmax-trained one (Wang et al., 2018a). SphereFace and L-Softmax remove the ReLU before the embedding layer, so that features are not confined to the non-negative orthant (SphereFace, arXiv appendix).

Variants

• Sub-center ArcFace (Deng et al., 2020; TPAMI version, Deng et al., 2022) gives each identity $K$ sub-centres and uses the closest one, $\theta_j = \arccos(\max_k W_{jk}^\top x_i)$, in the ArcFace loss. A noisy or very different image can then form its own sub-class instead of pulling the main one. After training, non-dominant sub-centres are dropped together with images more than a threshold angle ($75°$ in the TPAMI experiments) from their dominant sub-centre, and the model is retrained on the cleaned data. On MS1MV0 the TPAMI version reports a TAR at FAR $= 10^{-4}$ on IJB-C of 93.72% for $K = 3$ and 95.92% after dropping and retraining, against 90.27% for plain ArcFace and 96.50% for ArcFace on the cleaned MS1MV3 (all with ResNet50); $K = 10$ fell to 67.94%.
• CurricularFace (Huang et al., 2020) keeps ArcFace’s target logit and modulates the logits of hard negatives, those with $\cos\theta_j > \cos(\theta_{y_i} + m)$, to $\cos\theta_j\,(t + \cos\theta_j)$, where $t$ is an exponential moving average of the mean positive cosine, starting at 0 (the paper’s Eq. 9, as printed, weights the current batch by $\alpha = 0.99$). Early in training $t$ is near 0 and hard negatives are down-weighted; later they are emphasized.
• MagFace (Meng et al., 2021) keeps the cosine logits but makes the feature norm $a_i$, bounded to $[10, 110]$, an input of the loss: ArcFace’s margin becomes a function $m(a_i)$ that increases with the norm, and a regularizer $g(a_i)$ rewards large norms, so that the norm of the embedding becomes a measure of face quality.
• AdaFace (Kim, Jain & Liu, 2022) uses the feature norm, standardized with batch statistics and clipped to $[-1, 1]$, as a proxy for image quality, and moves between margins: its target logit $s\cos(\theta + g_{\text{angle}}) - g_{\text{add}}$, with $g_{\text{angle}} = -m\,\hat{z}$ and $g_{\text{add}} = m\,\hat{z} + m$ for the standardized norm $\hat{z}$, is ArcFace at $\hat{z} = -1$ and CosFace at $\hat{z} = 0$. Low-quality images thereby get less emphasis on hard samples.

Reported Results

The benchmarks, their protocols and the meaning of TAR at FAR are described in face recognition. Results from different papers differ in training data, network, alignment and test-time tricks such as concatenating the features of an image and its mirror (CosFace) or summing them (AM-Softmax), so only results within one table are comparable.

Controlled comparisons. SphereFace, trained on CASIA-WebFace (0.49 million images) with a 64-layer network, reports LFW accuracy rising with the margin from 97.88% for the original softmax to 99.42% for $m = 4$, and 95.0% on YouTube Faces; on MegaFace Challenge 1 with FaceScrub probes under the small-training-set protocol, a rank-1 identification rate of 72.729% with one million distractors and a TAR of 85.561% at FAR $= 10^{-6}$ (Liu et al., 2017). With the same data and network, CosFace reports 99.33% on LFW, 96.1% on YouTube Faces and 77.11% and 89.88% on the same two MegaFace measures (Wang et al., 2018a, Table 2). Trained on about 5 million images of more than 90,000 identities, partly from a private dataset, its single model reports 99.73% on LFW, 97.6% on YouTube Faces, and 82.72% and 96.65% on MegaFace under the large-training-set protocol (Tables 3 and 4). The CVPR ArcFace ablation (CASIA-WebFace, ResNet50; Deng et al., 2019, Table 2) gives verification accuracies on LFW, CFP-FP and AgeDB-30 of:

Loss	LFW	CFP-FP	AgeDB-30
Softmax	99.08	94.39	92.33
Normalized softmax ($s = 64$)	98.56	89.79	88.72
SphereFace, $m_1 = 1.35$	99.11	94.38	91.70
CosFace, $m_3 = 0.35$	99.51	95.44	94.56
Combined $(1, 0.3, 0.2)$	99.48	95.12	94.38
ArcFace, $m = 0.5$	99.53	95.56	95.15

The TPAMI version repeats the table with a normalized softmax at $s = 20$ added (99.20, 94.61, 92.65) and different values in the rows that combine the normalized softmax with intra- and inter-class losses. MagFace re-implemented the losses under one protocol (MS1MV2, ResNet100, the recommended hyperparameters of each) and reports TAR at FAR $= 10^{-4}$ on IJB-C of 91.77% for SphereFace, 95.56% for CosFace, 95.74% for ArcFace and 95.81% for MagFace; at FAR $= 10^{-6}$ on IJB-B, CosFace’s 40.41% was above ArcFace’s 38.68% (Meng et al., 2021, Table 2).

Headline results. The CVPR version reports, for ArcFace trained on MS1MV2 (5.8 million images of 85,000 identities, a version of MS-Celeb-1M refined by the authors) with ResNet100, 99.83% on LFW and 98.02% on YouTube Faces, and a TAR at FAR $= 10^{-4}$ of 0.942 on IJB-B and 0.956 on IJB-C; trained on VGGFace2 with ResNet50, 0.898 and 0.921 (Deng et al., 2019, Tables 4 and 7). The TPAMI version reports 89.8% and 92.79% for the VGGFace2 model (the CVPR table has 0.921 for IJB-C), 95.42% and 96.83% for ResNet100 on MS1MV3, and 96.02% and 97.27% on IBUG-500K, 11.96 million images of 493,000 identities cleaned automatically with sub-center ArcFace (Deng et al., 2022, Table 7). On MegaFace the ArcFace authors found many wrongly labelled images, manually refined the probe and distractor sets, and report both: for MS1MV2 and ResNet100 the CVPR version gives a rank-1 rate of 81.03% and a TAR at FAR $= 10^{-6}$ of 96.98% on the original data, and 98.35% and 98.48% on the refined data (Table 6). Numbers on the refined MegaFace are not comparable with numbers on the original.

Later variants. CurricularFace reports 94.8% and 96.1% on IJB-B and IJB-C at FAR $= 10^{-4}$ (MS1MV2, ResNet100) against the 94.2% and 95.6% printed for ArcFace (Huang et al., 2020, Table 5). AdaFace reports 95.67% and 96.89% in the same setting, and in its table lists ArcFace at 94.25% and 96.03% without saying where these differ from ArcFace’s printed values; its largest gains are on the low-quality IJB-S and TinyFace benchmarks (Kim, Jain & Liu, 2022, Table 3). CurricularFace, MagFace and the TPAMI ArcFace all describe LFW results as saturated or nearly so; the TPAMI version puts the level at about 99.8%. Differences of a few hundredths of a percentage point on a saturated benchmark are the situation discussed in benchmark overfitting.

Examples

Checking the ArcFace logit

The first script implements the combined margin for a batch of cosines and checks it against the formula. It builds 2,002 eight-dimensional features at known angles $\theta \in [0, \pi]$ to the weight of class 0, including $\theta = \pi - m$ and $\theta$ just below $\pi$, with arbitrary feature and weight norms, and compares the logits with $s\cos(\theta + m)$ computed from the angles directly and with the forward pass of the current official ArcFace class. It then tabulates the target logit and its derivative with respect to $\cos\theta$ for the formula, the 2018 fallback, the straight-through version (st, value of $\cos(\theta + m)$, gradient of $\cos\theta$) and the official class. PyTorch 2.14.1, float64, under a second.

import math
import torch
import torch.nn.functional as F

torch.set_num_threads(1)
torch.set_default_dtype(torch.float64)

def margin_target(cos, m1=1.0, m2=0.0, m3=0.0, guard="none"):
    """Target logit (before scaling) of the combined margin cos(m1*theta + m2) - m3.
    guard="fallback": insightface's 2018 rule, cos(theta) - m2*sin(m2) once theta >= pi - m2.
    guard="st": value cos(theta + m2), gradient as if the target logit were cos(theta)."""
    eps = torch.finfo(cos.dtype).eps                 # keeps arccos's gradient finite
    theta = torch.arccos(cos.clamp(-1 + eps, 1 - eps))
    t = torch.cos(m1 * theta + m2) - m3
    if guard == "fallback":
        t = torch.where(theta < math.pi - m2, t, cos - m2 * math.sin(m2))
    if guard == "st":
        t = cos + (t - cos).detach()
    return t

def margin_logits(cos, y, s, **kw):
    """Scaled logits: margin on the label's column, plain cosine elsewhere."""
    hot = F.one_hot(y, cos.shape[1]).bool()
    return s * torch.where(hot, margin_target(cos, **kw), cos)

class InsightFaceArcFace(torch.nn.Module):          # forward() of losses.ArcFace, commit c6c3af7
    def __init__(self, s=64.0, margin=0.5):
        super().__init__(); self.s, self.margin = s, margin
    def forward(self, logits, labels):
        index = torch.where(labels != -1)[0]
        target_logit = logits[index, labels[index].view(-1)]
        with torch.no_grad():
            target_logit.arccos_()
            logits.arccos_()
            final_target_logit = target_logit + self.margin
            logits[index, labels[index].view(-1)] = final_target_logit
            logits.cos_()
        return logits * self.s

# Features at exactly known angles theta to the weight of class 0 (8-D, two classes).
s, m = 64.0, 0.5
gen = torch.Generator().manual_seed(0)
theta = torch.cat([torch.rand(2000, generator=gen) * math.pi,
                   torch.tensor([math.pi - m, math.pi - 1e-6])])
x = torch.zeros(len(theta), 8); x[:, 0], x[:, 1] = theta.cos(), theta.sin()
x = x * (1 + 4 * torch.rand(len(theta), 1, generator=gen))      # arbitrary feature norms
W = torch.zeros(2, 8); W[0, 0], W[1, 2] = 3.0, 0.7                # arbitrary weight norms
cos = F.normalize(x) @ F.normalize(W).T
y = torch.zeros(len(theta), dtype=torch.long)
z = margin_logits(cos, y, s, m2=m)
print("PyTorch", torch.__version__)
print("max |z_target - s*cos(theta+m)|:", (z[:, 0] - s * torch.cos(theta + m)).abs().max().item())
print("max |z_other - s*cos(theta_2)| :", (z[:, 1] - s * cos[:, 1]).abs().max().item())
print("features with theta + m > pi   :", int((theta + m > math.pi).sum()))
z_if = InsightFaceArcFace(s, m)(cos.clone(), y)
print("max |ours - insightface|       :", (z - z_if).abs().max().item())

# Where the exact target logit stops decreasing, and the gradient with respect to cos(theta).
grid = torch.linspace(0, math.pi, 180001)
print("argmin_theta cos(theta+m) (deg):", round(math.degrees(grid[torch.cos(grid + m).argmin()].item()), 3))
print(" theta | target: exact fallback  CosFace | dt/dcos: exact fallback   st  insightface")
for deg in [10, 60, 90, 120, 140, 151, 152, 160, 170]:
    c = torch.tensor([math.cos(math.radians(deg))], requires_grad=True)
    row = []
    for g in ["none", "fallback", "st"]:
        t = margin_target(c, m2=m, guard=g)
        row.append((t.item(), torch.autograd.grad(t, c)[0].item()))
    c2 = c.detach().clone().view(1, 1).requires_grad_()
    InsightFaceArcFace(1.0, m)(c2 * 1.0, torch.tensor([0])).sum().backward()
    print(f"{deg:5d}  | {row[0][0]:13.3f} {row[1][0]:8.3f} {c.item() - 0.35:8.3f} |"
          f" {row[0][1]:14.3f} {row[1][1]:8.3f} {row[2][1]:5.2f} {c2.grad.item():10.2f}")

PyTorch 2.14.1
max |z_target - s*cos(theta+m)|: 2.0427108893272816e-09
max |z_other - s*cos(theta_2)| : 0.0
features with theta + m > pi   : 308
max |ours - insightface|       : 3.91886975727153e-15
argmin_theta cos(theta+m) (deg): 151.352
 theta | target: exact fallback  CosFace | dt/dcos: exact fallback   st  insightface
   10  |         0.781    0.781    0.635 |          3.597    3.597  1.00       1.00
   60  |         0.024    0.024    0.150 |          1.154    1.154  1.00       1.00
   90  |        -0.479   -0.479   -0.350 |          0.878    0.878  1.00       1.00
  120  |        -0.854   -0.854   -0.850 |          0.601    0.601  1.00       1.00
  140  |        -0.980   -0.980   -1.116 |          0.306    0.306  1.00       1.00
  151  |        -1.000   -1.000   -1.225 |          0.013    0.013  1.00       1.00
  152  |        -1.000   -1.123   -1.233 |         -0.024    1.000  1.00       1.00
  160  |        -0.989   -1.179   -1.290 |         -0.440    1.000  1.00       1.00
  170  |        -0.948   -1.225   -1.335 |         -1.841    1.000  1.00       1.00

The logits match the formula to rounding error over the whole range, including the 308 features with $\theta + m > \pi$, and are independent of the norms of $x$ and $W$; the largest error, $2 \times 10^{-9}$ on logits of size 64, comes from the feature at $\pi - 10^{-6}$, where $\arccos$ is badly conditioned. The official class gives the same forward values to $10^{-15}$. The table shows the three behaviours described under Implementation. The exact target logit bottoms out at $\pi - m = 151.35°$ and rises again, and its gradient follows $\sin(\theta + m)/\sin\theta$: 3.6 at $10°$, 0.88 at $90°$, almost 0 at $151°$ and negative beyond. The fallback is monotonic but jumps from $-1$ to $-\cos m - m\sin m = -1.117$ at $\pi - m$, after which its gradient is 1. The straight-through version and the official class have gradient 1 everywhere, like CosFace. Comparing the target columns, ArcFace’s penalty $\cos\theta - \cos(\theta + m) = 2\sin(\theta + m/2)\sin(m/2)$ exceeds CosFace’s constant 0.35 between about $31°$ and $121°$ (by our arithmetic) and is smaller outside, in line with the first ArcFace version’s observation that its curve lies below CosFace’s between $30°$ and $90°$ and above it below $30°$.

Margins on synthetic faces

The second script trains a small convolutional embedding network on synthetic faces with three losses and evaluates it on identities it never saw. Each identity is a vector of eight geometric parameters (head width and height, eye spacing, height and size, nose length, mouth width and height); each image is a $32 \times 32$ rendering with random in-plane rotation, shift, scale, smile width, eye opening, a lighting ramp, gain and noise, standardized to zero mean and unit variance, the same renderer as in the face recognition example. Training uses 200 identities with 10 fresh images each per epoch, 10 epochs of Adam, a 64-dimensional embedding and $s = 16$ for all losses (the CosFace bound gives about 7.5 for 200 classes at $P_W = 0.9$, by our arithmetic). The losses are the normalized softmax, CosFace with $m = 0.35$, and ArcFace with $m = 0.5$ in the straight-through form of the official code. The table reports the mean angle between new images of the training identities and their class weights, then, for all pairs among 10 images each of 150 new identities, the mean genuine and impostor angles, the angle of the threshold that gives FAR $= 10^{-3}$, and the TAR at that threshold. The three losses see identical identities, images and initial weights in each seed. It continues the first script, uses four threads and takes about 40 seconds.

# Continues the first block (margin_logits). Training runs in float32 on four threads.
import torch.nn as nn

torch.set_default_dtype(torch.float32)
torch.set_num_threads(4)
H = 32
yy, xx = torch.meshgrid(torch.arange(H) - 15.5, torch.arange(H) - 15.5, indexing="ij")

def render(p, gen):
    """One 32x32 face per row of p (8 identity parameters in [0, 1]): ellipse head, eyes,
    nose, mouth, under random rotation, shift, scale, expression, lighting and noise."""
    n = len(p)
    U = lambda lo, hi, *s: lo + (hi - lo) * torch.rand(n, *s, generator=gen)
    L = lambda k, lo, hi: (lo + (hi - lo) * p[:, k])[:, None, None]
    th, sc = torch.deg2rad(U(-8, 8))[:, None, None], U(0.94, 1.06)[:, None, None]
    sx, sy = U(-1.5, 1.5)[:, None, None], U(-1.5, 1.5)[:, None, None]
    x = ((xx - sx) * th.cos() + (yy - sy) * th.sin()) / sc
    y = (-(xx - sx) * th.sin() + (yy - sy) * th.cos()) / sc
    a, b = L(0, 8, 13), L(1, 10.5, 15)                          # head half-axes
    ex, ey, er = L(2, 3, 7), L(3, -7, -2), L(4, 1, 2.6)         # eye spacing, height, size
    nl, mw, my = L(5, 1, 5.5), L(6, 2.5, 7.5), L(7, 3.5, 8.5)   # nose, mouth width, height
    mw = mw * U(0.85, 1.15)[:, None, None]                      # expression: smile width
    lid = U(0.5, 1.0)[:, None, None]                            # expression: eye opening
    head = (x / a) ** 2 + (y / b) ** 2 < 1
    eyes = ((x.abs() - ex) / er) ** 2 + ((y - ey) / (er * lid)) ** 2 < 1
    nose = (x.abs() < 0.8) & (y > ey + 1) & (y < ey + 1 + nl)
    mouth = (x.abs() < mw) & ((y - my).abs() < 1.0)
    g = U(-0.03, 0.03, 2)[:, :, None, None]
    img = (U(0.1, 0.4)[:, None, None] + head * U(0.4, 0.6)[:, None, None]
           - (eyes | nose | mouth) * head * U(0.35, 0.5)[:, None, None]
           + g[:, 0] * xx + g[:, 1] * yy) * U(0.7, 1.3)[:, None, None]
    img = img + 0.06 * torch.randn(n, H, H, generator=gen)
    img = (img - img.mean((1, 2), keepdim=True)) / img.std((1, 2), keepdim=True)
    return img[:, None]

def embedder(d=64):
    return nn.Sequential(
        nn.Conv2d(1, 16, 3, stride=2, padding=1), nn.BatchNorm2d(16), nn.ReLU(),
        nn.Conv2d(16, 32, 3, padding=1), nn.BatchNorm2d(32), nn.ReLU(), nn.MaxPool2d(2),
        nn.Conv2d(32, 64, 3, padding=1), nn.BatchNorm2d(64), nn.ReLU(), nn.MaxPool2d(2),
        nn.Flatten(), nn.Linear(64 * 4 * 4, d))

def train(P, loss_kw, gen, s=16.0, per_id=10, epochs=10, lr=1e-2):
    """Identity classification with cosine logits; loss_kw selects the margin."""
    y = torch.arange(len(P)).repeat_interleave(per_id)
    f, W = embedder(), nn.Parameter(0.1 * torch.randn(len(P), 64, generator=gen))
    opt = torch.optim.Adam(list(f.parameters()) + [W], lr=lr)
    sched = torch.optim.lr_scheduler.OneCycleLR(opt, lr, total_steps=epochs * -(-len(y) // 128))
    for _ in range(epochs):
        X = render(P[y], gen)                                    # fresh images every epoch
        for idx in torch.randperm(len(y), generator=gen).split(128):
            cos = F.normalize(f(X[idx])) @ F.normalize(W).T
            loss = F.cross_entropy(margin_logits(cos, y[idx], s, **loss_kw), y[idx])
            opt.zero_grad(); loss.backward(); opt.step(); sched.step()
    return f.eval(), W.detach()

def evaluate(f, W, P_train, P_test, gen):
    deg = lambda c: torch.rad2deg(torch.arccos(c.clamp(-1, 1)))
    with torch.no_grad():
        ytr = torch.arange(len(P_train)).repeat_interleave(4)  # new images of training ids
        th_y = deg((F.normalize(f(render(P_train[ytr], gen))) * F.normalize(W)[ytr]).sum(1))
        ids = torch.arange(len(P_test)).repeat_interleave(10)   # unseen identities, all pairs
        E = F.normalize(f(render(P_test[ids], gen)))
    iu = torch.triu_indices(len(ids), len(ids), 1)
    sim, same = (E @ E.T)[iu[0], iu[1]], ids[iu[0]] == ids[iu[1]]
    imp = sim[~same]
    t = imp.topk(int(1e-3 * len(imp)) + 1).values[-1]          # threshold for FAR = 1e-3
    return [th_y.mean(), deg(sim[same]).mean(), deg(imp).mean(), deg(t), (sim[same] > t).float().mean()]

def run(variants, seeds=range(3)):
    print("seed  loss            th_y(train)  genuine  impostor  thr@1e-3  TAR@1e-3")
    for seed in seeds:
        for name, kw in variants:
            gen = torch.Generator().manual_seed(seed); torch.manual_seed(seed)
            P_train, P_test = torch.rand(200, 8, generator=gen), torch.rand(150, 8, generator=gen)
            r = evaluate(*train(P_train, kw, gen), P_train, P_test, gen)
            print(f"{seed:4d}  {name:14s}" + "".join(f"{v:10.1f}" for v in r[:4]) + f"{r[4]:10.3f}")

run([("norm-softmax", {}), ("CosFace 0.35", dict(m3=0.35)), ("ArcFace 0.5 st", dict(m2=0.5, guard="st"))])

seed  loss            th_y(train)  genuine  impostor  thr@1e-3  TAR@1e-3
   0  norm-softmax        45.0      28.3      89.4      27.3     0.504
   0  CosFace 0.35        42.2      28.9      89.4      28.6     0.535
   0  ArcFace 0.5 st      42.2      28.9      89.4      28.6     0.535
   1  norm-softmax        48.9      28.2      89.0      24.8     0.389
   1  CosFace 0.35        46.8      28.5      89.1      25.6     0.407
   1  ArcFace 0.5 st      46.8      28.6      89.1      25.6     0.407
   2  norm-softmax        49.7      29.2      88.5      26.7     0.440
   2  CosFace 0.35        48.0      29.6      88.6      27.6     0.459
   2  ArcFace 0.5 st      47.9      29.8      88.6      27.9     0.463

Running the script twice gives identical output. In all three seeds both margins pull the training features 1.7 to 2.8 degrees closer to their class weights than the normalized softmax and raise the TAR at FAR $= 10^{-3}$ on the new identities by 0.018 to 0.031. The gain does not come from smaller genuine angles, which grow slightly, but from the impostor tail: the threshold angle that admits one impostor pair in a thousand moves out by 0.8 to 1.3 degrees, so more genuine pairs fall inside it. The spread between seeds (TAR 0.389 to 0.504 for the normalized softmax) is larger than the effect of the margin, which is consistent within each seed because the three runs share data and initialization. CosFace and the straight-through ArcFace end up almost identical. By our reasoning this is expected: for both, the derivative of the loss with respect to the target cosine is $s(p_{y_i} - 1)$, so they differ only through the predicted probability of the true class, and the two margins are of similar size at the angles reached here. The example is small and synthetic; it shows the mechanism, not the size of the effect on real faces.

The formula with plain autograd

The third script reruns the same seeds with ArcFace computed as written, with automatic differentiation through $\arccos$, and with the 2018 fallback. It continues the second and takes about 27 seconds.

# Continues the second block: the formula with plain autograd, and insightface's 2018 guard.
run([("ArcFace 0.5", dict(m2=0.5)), ("ArcFace 0.5 fb", dict(m2=0.5, guard="fallback"))])

seed  loss            th_y(train)  genuine  impostor  thr@1e-3  TAR@1e-3
   0  ArcFace 0.5         31.7      25.5      81.6      25.4     0.541
   0  ArcFace 0.5 fb      31.7      25.5      81.6      25.4     0.541
   1  ArcFace 0.5        143.2       4.9      10.6       3.9     0.213
   1  ArcFace 0.5 fb     143.2       4.9      10.6       3.9     0.213
   2  ArcFace 0.5        153.9       3.1       4.5       2.1     0.056
   2  ArcFace 0.5 fb     152.0       3.4       6.1       2.5     0.119

In seed 0 the exact gradient, which exceeds 1 below $75.7°$, pulled the training features much closer to their class weights (31.7 degrees, against 42.2 for the straight-through version) and gave the highest TAR of all the runs, 0.541. In seeds 1 and 2 training collapsed: new images of the training identities ended on average $143°$ and $154°$ from their own class weights, impostor pairs of new identities were on average only 10.6 and 4.5 degrees apart, and the TAR dropped to 0.213 and 0.056. Seed 2’s average lies beyond $\pi - m$, where the exact gradient points away from the class weight; seed 1’s lies below it, where the gradient is positive but small (0.31 at $140°$ and 0.013 at $151°$ in the first table), and there the fallback, which only acts beyond $\pi - m$, changed nothing. In seed 2 the fallback held the average at $152°$, just past $\pi - m$, instead of $154°$, and training still failed. Our reading is that once features and class weights drift far apart early in training, the shrinking factor $\sin(\theta + m)/\sin\theta$ weakens the pull towards the class weight while the push away from the other classes keeps its strength, so the drift feeds itself; the straight-through gradient of the current official code has no such factor and did not collapse in any seed. The first ArcFace version’s argument that angles beyond $\pi - m$ are almost never reached rests on the angles seen in its own training runs; in this small synthetic setup they were reached, and in seed 1 training collapsed below $\pi - m$, where the 2018 guard does not act.

Common Misconceptions

• “ArcFace is a network.” It is a loss. The models released and evaluated as “ArcFace” are ResNets and other networks trained with it; at test time the classifier, and with it the margin, is discarded, and faces are compared by the cosine of their embeddings.
• “An additive angular margin and an additive cosine margin behave very differently.” At the angles where training happens the two target logits are close, and in controlled comparisons the results are within a few tenths of a percentage point: 99.51% against 99.53% on LFW in the ArcFace ablation, 95.56% against 95.74% TAR at FAR $= 10^{-4}$ on IJB-C in MagFace’s re-implementation, where CosFace was ahead at FAR $= 10^{-6}$ on IJB-B. The geometric difference is real; its effect on accuracy is small.
• “$\cos(\theta + m)$ is always a penalty.” Only for $\theta \le \pi - m$. Beyond it the target logit rises again, and an implementation that differentiates through $\arccos$ pushes such features away from their class (see Implementation).
• “A larger scale is always better.” The scale must exceed a lower bound for the loss to approach zero, but the normalized softmax lost accuracy at $s = 64$ compared with $s = 20$ in the TPAMI ablation.
• “The class weights are the class centres.” They are learned directions that only approximately match the mean of each class’s features; the gap was $44°$ under the normalized softmax and $14°$ under ArcFace in the CVPR angle statistics.
• “Higher LFW accuracy means a better loss.” LFW is saturated. AM-Softmax at $m = 0.35$ scored 98.98% on LFW, below the 99.08% listed for A-Softmax in the same table, while improving MegaFace rank-1 from 67.41% to 72.47% (Wang et al., 2018b, Table 1).

Where It Is Used

• Face recognition. Margin-based softmax losses are the training objective of the deep face recognizers described in face recognition: SphereFace, CosFace, ArcFace and their successors all report results on its benchmarks. The official InsightFace training code uses ArcFace by default and CosFace in its WebFace and Glint360K configurations.
• Deep metric learning. Classification with normalized weights as proxies is a baseline for retrieval embeddings, compared with pair and triplet losses in metric learning, which also lists library implementations of the CosFace and ArcFace losses.
• Face quality. MagFace trains the norm of the embedding to measure face quality, and AdaFace uses the feature norm as a proxy for image quality during training.
• Alternatives. The triplet loss trains the same kind of embedding from triplets, as FaceNet did; the margin losses avoid its sampling problem but need one weight vector per training identity. How embeddings are compared at test time is covered in similarity and distance measures.