Focal Loss

The focal loss is a reshaped cross-entropy loss that multiplies the loss of each example by $(1 - p_t)^{\gamma}$, where $p_t$ is the probability the model gives the correct label. Well-classified examples, with $p_t$ close to 1, have their loss scaled down sharply. Badly classified examples keep almost all of theirs. Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He and Piotr Dollár introduced it in “Focal Loss for Dense Object Detection” (ICCV 2017). A dense detector scores some 100,000 candidate locations per image, and almost all of them are easy background. Summed together, their small cross-entropy losses swamp the few objects. The focal loss keeps every example but lets the easy ones contribute little. With it, a plain one-stage detector, RetinaNet, surpassed the accuracy of the two-stage detectors of the time. It has since been adopted as the classification loss of many dense detectors and is used in segmentation, keypoint heatmaps and multi-label classification.

The RetinaNet article covers the detector: the paper’s ablations of $\alpha$ and $\gamma$, the prior initialization of the classifier bias, the alternative form $\mathrm{FL}^*$ from the arXiv version’s appendix, and a training experiment. This article covers the loss itself: its definition and parameters, what it does to the gradient and to the probabilities a model learns, and the variants built on it.

Definition

For a binary label and a predicted probability $p$ of the positive class, the focal loss is the binary cross-entropy $-\log p_t$ multiplied by a modulating factor $(1 - p_t)^{\gamma}$, where $\gamma \ge 0$ is the focusing parameter. With $\gamma = 0$ it is the cross-entropy. In practice an extra class weight $\alpha_t$ is added, giving the α-balanced focal loss. Lin et al. used it in their experiments because it was slightly more accurate than the unweighted form.

Intuition

Under cross-entropy, a background location that the detector already rejects with 99% confidence still costs $-\log 0.99 \approx 0.01$. That is small, but a detector with 100,000 such locations and a few dozen objects collects most of its loss and gradient from them. Lin et al. identify this class imbalance as the main obstacle for one-stage detectors: training is inefficient because the easy negatives contribute no useful learning signal, and en masse they “can overwhelm training and lead to degenerate models”.

There are two classic remedies. A class weight $\alpha$ raises the importance of the rare class. Sampling, as in hard negative mining, throws most negatives away. The focal loss does something else. It looks at how well each example is already classified and turns down the loss of the easy ones, whichever class they belong to. Lin et al. describe it as the opposite of a robust loss such as the Huber loss. A robust loss down-weights examples with large errors, the outliers. The focal loss down-weights the inliers, the easy examples, so that their total stays small however many there are.

Formal Definition

Binary form

Lin et al. write the label as $y \in \{\pm 1\}$ and the model’s probability for $y = 1$ as $p$, usually $p = \sigma(x)$ for a logit $x$. They define

$$p_t = \begin{cases} p & \text{if } y = 1, \\ 1 - p & \text{otherwise,} \end{cases} \qquad \mathrm{CE}(p_t) = -\log p_t .$$

The α-balanced cross-entropy weights class 1 by $\alpha \in [0, 1]$ and class $-1$ by $1 - \alpha$, with $\alpha_t$ defined like $p_t$. The focal loss and its α-balanced form are (their Eqs. 4 and 5)

$$\mathrm{FL}(p_t) = -(1 - p_t)^{\gamma} \log p_t , \qquad \mathrm{FL}(p_t) = -\alpha_t (1 - p_t)^{\gamma} \log p_t .$$

The ratio of the focal loss to the cross-entropy is just the modulating factor $(1 - p_t)^{\gamma}$. With $\gamma = 2$, the paper’s default, an example with $p_t = 0.9$ gets 100 times less loss than under cross-entropy, and one with $p_t \approx 0.968$ about 1,000 times less. An example with $p_t \le 0.5$ loses at most a factor of 4. Lin et al. note that their loss layer combines the sigmoid with the loss for numerical stability. Implementations compute $-\log p_t$ from the logit, as binary_cross_entropy_with_logits does, and form the factor from the sigmoid. TorchVision’s sigmoid_focal_loss (v0.29.2), which its source says was taken from fvcore, does this, with defaults $\alpha = 0.25$, $\gamma = 2$ and per-element losses unless a reduction is requested.

Multi-class form

The paper studies the binary loss, and a footnote says the multi-class extension is straightforward and works well. RetinaNet uses $K$ independent sigmoid outputs per anchor, so the binary loss is applied to each class of each anchor separately. For a softmax output $\mathbf{u}$ and a one-hot or soft target $\mathbf{v}$, Charoenphakdee, Vongkulbhisal, Chairatanakul and Sugiyama (2021, Eq. 5) write the multi-class focal loss as

$$\ell_{\mathrm{FL}}^{\gamma}(\mathbf{u}, \mathbf{v}) = -\sum_{i=1}^{K} v_i (1 - u_i)^{\gamma} \log u_i ,$$

which reduces to the softmax cross-entropy at $\gamma = 0$. For $K = 2$ it is the binary loss above without $\alpha$ (our observation).

Normalization and initialization

Two training details belong with the loss in RetinaNet, and the RetinaNet article covers both. First, the focal loss is summed over all anchors of an image and divided by the number of anchors assigned to an object, not by the number of anchors. The authors’ reason is that most anchors are easy negatives whose loss is negligible under the focal loss. Second, the bias of the last classification layer is set to $-\log\big((1 - \pi)/\pi\big)$, so that every anchor starts with foreground probability $\pi = 0.01$. Lin et al. present this as a change of initialization, not of the loss, and found that it stabilized early training for both cross-entropy and the focal loss under heavy imbalance. CornerNet, for instance, follows Lin et al. in setting the biases of its heatmap layers.

Properties

The gradient

Writing $x_t = yx$, so that $p_t = \sigma(x_t)$, the derivative of the cross-entropy with respect to $x_t$ is $-(1 - p_t)$ (our derivation). Differentiating $-(1 - p_t)^{\gamma} \log p_t$ with $\mathrm{d}p_t/\mathrm{d}x_t = p_t(1 - p_t)$ gives (our derivation)

$$\frac{\partial\,\mathrm{FL}}{\partial x_t} = (1 - p_t)^{\gamma} \bigl(\gamma\, p_t \log p_t + p_t - 1\bigr).$$

As $p_t \to 1$ the bracket behaves like $-(\gamma + 1)(1 - p_t)$, so an easy example’s gradient falls off as $(1 - p_t)^{\gamma + 1}$ instead of $1 - p_t$. Multiplied by $y$, it is the derivative with respect to the logit $x$ that the RetinaNet article quotes from the arXiv version. The one-stage object detectors article measures the consequence on a fixed objectness map: it compares the share of the total loss and of the gradient that comes from easy negatives under cross-entropy and under the focal loss. The RetinaNet article shows how the loss distributes over sorted negatives in Lin et al.’s analysis of a converged detector.

α and γ interact

The two parameters do different jobs. $\alpha$ weights the classes, and Lin et al. note that it does not separate easy from hard examples. $\gamma$ weights examples by difficulty. They must be chosen together. In the paper’s COCO ablation the best $\alpha$ fell as $\gamma$ rose, and with $\gamma = 2$ the best value was $\alpha = 0.25$. That value gives the rare foreground class a smaller weight than the background. The authors’ explanation is that once the easy negatives are down-weighted, the positives need less emphasis. The RetinaNet article gives the table.

The minimiser and calibration

A loss used to train a probabilistic classifier is judged by two properties. It is classification-calibrated when minimizing it yields the Bayes-optimal decision. It is strictly proper when its expected value is minimized only by the true class probability, so that the learned scores are honest probabilities. Cross-entropy has both properties. Charoenphakdee et al. (CVPR 2021) analysed the focal loss in the multi-class form above, which has no $\alpha$:

• Classification-calibrated (their Theorem 3). For any $\gamma \ge 0$ the focal loss is classification-calibrated. The minimizer of the expected loss preserves the order of the true class probabilities, so taking the arg max gives the Bayes classifier.
• Not strictly proper (Theorem 5). For every $\gamma > 0$ the minimizer is not, in general, the true class-posterior probability.
• Mostly under-confident. The minimizer is under-confident whenever its largest score lies between 0.5 and 1 (Corollary 9). In the binary case it is always under-confident unless the minimizer is uniform or one-hot (Corollary 10), and then it equals the posterior (Proposition 6). With many classes and small $\gamma$ it can also be over-confident; their Figure 2b shows this for 1,000 classes.
• Recoverable (Theorem 11). A closed-form transformation recovers the true posterior from the minimizer: $\eta_i = h(v_i) / \sum_l h(v_l)$ with $h(v) = v / \big[(1 - v)^{\gamma} - \gamma (1 - v)^{\gamma - 1} v \log v\big]$. It does not change the arg max (their Proposition 12).

None of these results covers the α-balanced form that RetinaNet trains with. The example below computes that case directly.

The practical picture is mixed. Charoenphakdee et al. trained ResNet-110 classifiers on SVHN and CIFAR-10 with one-hot labels and $\gamma = 1, 2, 3$ (their Figure 4). On SVHN the raw focal-loss networks had expected calibration errors of 2.57, 8.12 and 13.23 (in percent), and their transformation reduced them to 0.59, 0.55 and 0.62. On CIFAR-10 the raw networks had 2.20, 1.23 and 4.08, and the transformation made them worse (3.85, 3.67 and 3.71). The authors suggest that the transformation fails when a network does not reach the focal-risk minimizer, and hypothesize two reasons: the network overfits the one-hot labels, or the data are too few. Mukhoti et al. (NeurIPS 2020; not read for this article) reported, as Charoenphakdee et al. summarize them, that training with the focal loss without post-processing can give a lower ECE than cross-entropy. Charoenphakdee et al. find that this is not always the case (SVHN, for example). They suggest that it can happen when a classifier, especially a deep network, is over-confident: since the focal loss tends toward under-confidence, there may be a $\gamma > 0$ at which the two effects cancel and the ECE is lowest. Expected calibration error is defined and discussed in its own article.

Relation to sampling

The focal loss and hard negative mining both concentrate training on hard examples. Lin et al. contrast them: online hard example mining discards the easy examples completely, while the focal loss keeps them with small weights. In their ResNet-101 experiments on COCO, the best mining setting reached 32.8 AP on minival against 36.0 for the focal loss. The paper adds that the fixed 1:3 sampling ratio of two-stage detectors acts like an implicit α-balancing factor implemented by sampling.

Examples

What score does the focal loss ask for?

The theory above concerns the minimizer of the expected loss. The minimizer can be computed exactly, with no network, so its effect on a pixel classifier can be measured against ground truth. Take one binary output, a pixel’s “object” score or one class of one anchor in a sigmoid head, whose true probability of being positive is $q$. The expected α-balanced focal loss of a score $p$ is

$$E(p; q) = -q\, a_1 (1 - p)^{\gamma} \log p \;-\; (1 - q)\, a_0\, p^{\gamma} \log(1 - p),$$

with $a_1 = \alpha$ and $a_0 = 1 - \alpha$, or $a_1 = a_0 = 1$ without $\alpha$. Setting $\mathrm{d}E/\mathrm{d}p = 0$ gives $q\, a_1 A(p) = (1 - q)\, a_0 B(p)$, where

$$A(p) = \frac{(1 - p)^{\gamma}}{p} - \gamma (1 - p)^{\gamma - 1} \log p, \qquad B(p) = \frac{p^{\gamma}}{1 - p} - \gamma\, p^{\gamma - 1} \log(1 - p).$$

Both terms are positive, so the condition can be solved for $q$ in closed form, $q = a_0 B(p) / \big(a_1 A(p) + a_0 B(p)\big)$. Without $\alpha$ this is Charoenphakdee et al.’s transformation for $K = 2$. The version with $\alpha$ is our extension, and it follows from the same condition. The minimizer $p^*$ itself has no closed form, so the code finds it by bisection on the logit.

The data are a $256 \times 256$ synthetic image of 14 discs, the objects (intensity 1), on a background of 0, with Gaussian noise of standard deviation 0.5. Because the image is generated from a known model, every pixel’s exact posterior is known: $q = P(\text{object} \mid \text{intensity})$, with the realised object fraction as the prior. The “classifier” for each loss is that loss’s exact minimizer applied to $q$ at every pixel. Cross-entropy then returns $q$ itself, the best a classifier can do from intensity alone.

The settings are as follows:

• Without $\alpha$: cross-entropy and the focal loss with $\gamma = 0.5, 1, 2, 5$. These answer Charoenphakdee et al.’s question: what does $\gamma$ alone do to the scores?
• With $\alpha$: α-balanced cross-entropy with $\alpha = 0.75$, the best $\alpha$ for cross-entropy in Lin et al.’s ablation, and the focal loss with $\gamma = 2$, $\alpha = 0.25$, RetinaNet’s setting. These answer a different question: what does the loss that detectors actually train with do?

The script prints:

• the minimizer for a range of $q$, and the $q$ at which each loss’s minimizer equals 0.5;
• for every pixel, the gap $|p^* - q|$, mean confidence, accuracy and top-label ECE (15 equal-width bins) against the ground-truth mask;
• the number of pixels whose decision $p^* > 0.5$ differs from the Bayes decision $q > 0.5$;
• the error of the closed-form inverse;
• checks of the solver against SciPy’s brentq and a brute-force grid search, of the derivative against finite differences, and of the minimizer’s uniqueness.

It runs with NumPy 2.3.5 and SciPy 1.16.3 in a few seconds.

# What score does the focal loss ask for?  For a pixel whose true probability of "object" is q,
# the score p that minimises the expected loss  q*a1*FL(p) + (1-q)*a0*FL(1-p)  is computed exactly,
# for several gamma and alpha, and applied to the exact posterior of every pixel of a synthetic image.
import numpy as np
from scipy.optimize import brentq

# --- synthetic image with known ground truth: 14 discs (mean 1) on background (mean 0), noise 0.5
H = W = 256
rng = np.random.default_rng(0)
yy, xx = np.mgrid[0:H, 0:W]
mask = np.zeros((H, W), bool)
for _ in range(14):
    cy, cx = rng.integers(16, H - 16, 2)
    mask |= np.hypot(yy - cy, xx - cx) < rng.uniform(5, 12)
SIG = 0.5
img = mask.astype(float) + rng.normal(0, SIG, (H, W))
prior = mask.mean()
q = 1 / (1 + np.exp(-(np.log(prior / (1 - prior)) + (img - 0.5) / SIG**2)))   # exact P(object | intensity)
q, y = q.ravel(), mask.ravel()

def ece(p1, y, bins=15):                       # top-label expected calibration error, equal-width bins
    conf = np.maximum(p1, 1 - p1); correct = (p1 > 0.5) == y
    idx = np.minimum((conf * bins).astype(int), bins - 1)
    return sum((idx == b).mean() * abs(correct[idx == b].mean() - conf[idx == b].mean())
               for b in np.unique(idx))

print(f"image {H}x{W}: object fraction {prior:.4f}, pixels with q < 0.01: {np.mean(q < 0.01):.3f}, "
      f"with 0.1 < q < 0.9: {np.mean((q > 0.1) & (q < 0.9)):.3f}")
print(f"label check: ECE of the exact posterior q against the mask {ece(q, y):.4f}")

# --- stationarity of the expected loss: dE/dp = -q*a1*A(p) + (1-q)*a0*B(p), with A, B > 0
def AB(p, gam):
    pm = 1 - p
    A = pm**gam / p - (gam * pm**(gam - 1) * np.log(p) if gam > 0 else 0)
    B = p**gam / pm - (gam * p**(gam - 1) * np.log(pm) if gam > 0 else 0)
    return A, B

def dE(p, q, gam, a1, a0):
    A, B = AB(p, gam)
    return -q * a1 * A + (1 - q) * a0 * B

def E(p, q, gam, a1, a0):
    return -q * a1 * (1 - p)**gam * np.log(p) - (1 - q) * a0 * p**gam * np.log(1 - p)

def minimiser(q, gam, a1, a0):                 # vectorised bisection on the logit, 200 halvings
    lo, hi = np.full(np.shape(q), -30.0), np.full(np.shape(q), 30.0)
    for _ in range(200):
        mid = (lo + hi) / 2
        down = dE(1 / (1 + np.exp(-mid)), q, gam, a1, a0) < 0      # loss still falling: go right
        lo, hi = np.where(down, mid, lo), np.where(down, hi, mid)
    return 1 / (1 + np.exp(-(lo + hi) / 2))

def recover(p, gam, a1, a0):                   # closed-form inverse: the q whose minimiser is p
    A, B = AB(p, gam)
    return a0 * B / (a1 * A + a0 * B)

settings = [("CE", 0, 1, 1), ("FL g=0.5", 0.5, 1, 1), ("FL g=1", 1, 1, 1), ("FL g=2", 2, 1, 1),
            ("FL g=5", 5, 1, 1), ("CE a=0.75", 0, 0.75, 0.25), ("FL g=2 a=0.25", 2, 0.25, 0.75)]
qs = np.array([1e-4, 1e-3, 0.01, 0.1, 0.3, 0.5, 0.7, 0.9, 0.99])

print("\nminimising score p* for true probability q")
print("setting         " + "".join(f"{v:>8g}" for v in qs) + "   q for p*=0.5")
for name, gam, a1, a0 in settings:
    ps = minimiser(qs, gam, a1, a0)
    print(f"{name:15} " + "".join(f"{v:8.4f}" for v in ps) + f"   {recover(0.5, gam, a1, a0):10.4f}")

print("\nexact minimiser at every pixel, scored against the mask")
print("setting          mean|p*-q|  mean conf.  accuracy   ECE     decisions != (q>0.5)   max|recover-q|")
for name, gam, a1, a0 in settings:
    p = minimiser(q, gam, a1, a0)
    conf = np.maximum(p, 1 - p).mean(); acc = ((p > 0.5) == y).mean()
    print(f"{name:15} {np.abs(p - q).mean():10.4f} {conf:11.4f} {acc:9.4f} {ece(p, y):8.4f}"
          f" {int(((p > 0.5) != (q > 0.5)).sum()):12d} {np.abs(recover(p, gam, a1, a0) - q).max():21.1e}")

# --- checks: two other solvers, the analytic derivative, and uniqueness of the minimiser
d_brent = d_grid = d_fd = 0.0; changes = set()
grid = np.linspace(1e-6, 1 - 1e-6, 999999)
pfd = np.linspace(0.01, 0.99, 99)
for name, gam, a1, a0 in settings:
    for qq in qs:
        pb = minimiser(np.array(qq), gam, a1, a0)
        xb = brentq(lambda x: dE(1 / (1 + np.exp(-x)), qq, gam, a1, a0), -30, 30, xtol=1e-14)
        d_brent = max(d_brent, abs(pb - 1 / (1 + np.exp(-xb))))
        d_grid = max(d_grid, abs(pb - grid[np.argmin(E(grid, qq, gam, a1, a0))]))
        fd = (E(pfd + 1e-6, qq, gam, a1, a0) - E(pfd - 1e-6, qq, gam, a1, a0)) / 2e-6
        an = dE(pfd, qq, gam, a1, a0)
        d_fd = max(d_fd, np.max(np.abs(fd - an) / np.maximum(np.abs(an), 1e-3)))
        s = np.sign(dE(1 / (1 + np.exp(-np.linspace(-30, 30, 100001))), qq, gam, a1, a0))
        changes.add(int(np.sum(s[1:] != s[:-1])))
print(f"\nchecks: bisection vs brentq {d_brent:.1e}, vs grid search {d_grid:.1e}; "
      f"derivative vs finite differences {d_fd:.1e}; sign changes of dE/dp {sorted(changes)}")

Output:

image 256x256: object fraction 0.0478, pixels with q < 0.01: 0.551, with 0.1 < q < 0.9: 0.108
label check: ECE of the exact posterior q against the mask 0.0011

minimising score p* for true probability q
setting           0.0001   0.001    0.01     0.1     0.3     0.5     0.7     0.9    0.99   q for p*=0.5
CE                0.0001  0.0010  0.0100  0.1000  0.3000  0.5000  0.7000  0.9000  0.9900       0.5000
FL g=0.5          0.0016  0.0077  0.0359  0.1673  0.3489  0.5000  0.6511  0.8327  0.9641       0.5000
FL g=1            0.0072  0.0229  0.0730  0.2256  0.3828  0.5000  0.6172  0.7744  0.9270       0.5000
FL g=2            0.0335  0.0727  0.1536  0.3058  0.4215  0.5000  0.5785  0.6942  0.8464       0.5000
FL g=5            0.1607  0.2244  0.3047  0.4026  0.4620  0.5000  0.5380  0.5974  0.6953       0.5000
CE a=0.75         0.0003  0.0030  0.0294  0.2500  0.5625  0.7500  0.8750  0.9643  0.9966       0.2500
FL g=2 a=0.25     0.0231  0.0503  0.1084  0.2266  0.3260  0.3988  0.4765  0.6012  0.7862       0.7500

exact minimiser at every pixel, scored against the mask
setting          mean|p*-q|  mean conf.  accuracy   ECE     decisions != (q>0.5)   max|recover-q|
CE                  0.0000      0.9611    0.9608   0.0011            0               3.3e-16
FL g=0.5            0.0274      0.9338    0.9608   0.0272            0               2.8e-16
FL g=1              0.0603      0.9008    0.9608   0.0600            0               3.9e-16
FL g=2              0.1275      0.8336    0.9608   0.1272            0               5.0e-16
FL g=5              0.2621      0.6990    0.9608   0.2618            0               1.0e-15
CE a=0.75           0.0463      0.9280    0.9524   0.0247         1863               7.8e-16
FL g=2 a=0.25       0.0869      0.8770    0.9582   0.0830          801               5.6e-16

checks: bisection vs brentq 5.6e-16, vs grid search 4.8e-07; derivative vs finite differences 3.3e-06; sign changes of dE/dp [1]

The data line and the label check confirm the set-up. About 5% of the pixels are object, so the image is imbalanced, though far less than a detector. More than half of the pixels have $q < 0.01$, the easy negatives, and about 11% are genuinely ambiguous. The exact posterior is calibrated against the realised mask up to sampling noise, an ECE of 0.0011. The checks show that the three solvers agree, that the analytic derivative is right, and that the expected loss has a single minimum in every setting.

Without $\alpha$, every focal minimizer lies between $q$ and 0.5: the scores are pulled toward the middle, more strongly as $\gamma$ grows. This is the binary under-confidence of Corollary 10. The pull is largest in relative terms on the easy examples. With $\gamma = 2$ a pixel whose true object probability is 0.001 receives a score of 0.073, more than 70 times too high, and a pixel with $q = 0.9$ receives 0.69. On the image, mean confidence falls further below the accuracy as $\gamma$ grows, and the ECE rises from the noise floor to 0.127 at $\gamma = 2$ and 0.262 at $\gamma = 5$. The ECE nearly equals the mean gap $|p^* - q|$ because every pixel errs in the same direction. Yet the accuracy is identical in all five rows and no pixel’s decision changes: the minimizer is 0.5 exactly when $q$ is 0.5. That is the classification-calibration of Theorem 3, which holds for the loss without $\alpha$. Every row’s scores map back to $q$ through the closed-form inverse to within rounding.

With $\alpha$ the picture changes in two ways. First, the decision threshold moves. The minimizer equals 0.5 where $q = 1 - \alpha$, because $A(0.5) = B(0.5)$, so the threshold does not depend on $\gamma$. α-balanced cross-entropy with $\alpha = 0.75$ calls a pixel an object once its true probability exceeds 0.25. RetinaNet’s $\alpha = 0.25$ asks for 0.75. Both move away from the Bayes decision and lose accuracy against the mask (1,863 and 801 changed pixels). Second, $\alpha = 0.25$ partly cancels the upward pull of $\gamma$ on the background. At $q = 0.001$ the score falls from 0.073 to 0.050, and since most pixels are background, the ECE of the RetinaNet setting (0.083) is lower than that of $\gamma = 2$ alone. Its errors now point both ways, though: scores are still too high for background pixels and too low for likely objects (0.60 at $q = 0.9$). The closed-form inverse with $\alpha$ still recovers $q$ exactly. Two cautions apply. In our reading, the scores of a detector trained with $\gamma = 2$ and $\alpha = 0.25$ should not be read as probabilities, and its threshold of 0.05 corresponds roughly to a true probability of 0.001 for an exact minimizer (the printed 0.0503 at $q = 0.001$). Real networks do not reach the exact minimizer, and as Charoenphakdee et al.’s CIFAR-10 results show, the actual miscalibration of a trained network can go either way.

Variants

FL* and other forms. Lin et al. state that the exact form of the loss is not crucial. Their online appendix tests other instantiations that down-weight well-classified examples, and the RetinaNet article describes the $\mathrm{FL}^*$ form and its results. A hinge loss on $p_t$, tried in early experiments, was unstable.

Penalty-reduced focal loss for heatmaps. CornerNet (Law and Deng, 2018) detects box corners as peaks of per-class heatmaps. Each corner is one positive location, but negatives near it still give good boxes, so their penalty is reduced. The target $y_{cij}$ is an unnormalized Gaussian around each corner, and the loss (their Eq. 1) is

$$L_{\det} = -\frac{1}{N} \sum_{c,i,j} \begin{cases} (1 - p_{cij})^{\alpha} \log p_{cij} & \text{if } y_{cij} = 1, \\ (1 - y_{cij})^{\beta} (p_{cij})^{\alpha} \log(1 - p_{cij}) & \text{otherwise,} \end{cases}$$

with $N$ the number of objects, $\alpha = 2$ and $\beta = 4$. Here $\alpha$ plays the role of the focal $\gamma$, not of Lin et al.’s class weight. In CornerNet’s ablation on COCO validation images (their Table 2), training without the penalty reduction gave 32.9 AP, a fixed radius 35.6, and the object-dependent radius 38.5. CenterNet in the “Objects as Points” form uses the same loss for object centres, and its article works through the cost of a near miss.

Quality targets: GFL and Varifocal loss. The focal loss assumes 0/1 targets, so a detector’s class score says nothing about how well its box fits. Generalized Focal Loss (Li et al., 2020) trains the class branch to predict a joint score of class and localization quality. For the ground-truth class the target is the IoU of the predicted box with the ground truth, and 0 otherwise, as GFL V2 (Li et al., 2021) restates it. As the RetinaNet article gives GFL’s quality focal loss, the modulating factor becomes $|y - \sigma|^{\beta}$ for a target $y \in [0, 1]$. The GFL paper itself, a NeurIPS paper, was not read for this article. Varifocal loss (Zhang, Wang, Dayoub and Sünderhauf, CVPR 2021) also trains the class score $p$ toward an IoU-aware target $q$, but treats positives and negatives asymmetrically (their Eq. 2):

$$\mathrm{VFL}(p, q) = \begin{cases} -q\big(q \log p + (1 - q) \log(1 - p)\big) & q > 0, \\ -\alpha\, p^{\gamma} \log(1 - p) & q = 0 . \end{cases}$$

Only negatives are down-weighted. Positives are weighted by their target, so that high-quality positives count most. Replacing the focal loss with VFL in RetinaNet, FoveaBox, RepPoints and ATSS (ResNet-50, 1x schedule, COCO val2017) raised AP by 0.9 to 1.4 points, and the authors report VFL ahead of GFL in each of those detectors (their Table 5). For RetinaNet the figures were 36.5 AP with the focal loss, 37.3 with GFL and 37.4 with VFL.

Asymmetric loss for multi-label classification. A multi-label image classifier faces the same imbalance as a detector: an image has a few positive labels and many negative ones. On MS-COCO, with 2.9 labels per image out of 80, Ridnik et al. (ICCV 2021) report a positive–negative ratio of 0.0376. They found the focal loss better than cross-entropy for multi-label classification on MS-COCO (85.1% against 84.0% mAP, best for $2 \le \gamma \le 4$; their Figure 5). With a single $\gamma$, however, a value high enough to suppress easy negatives can also eliminate the gradients from the rare positives. Their asymmetric loss uses separate focusing parameters $\gamma_+$ and $\gamma_-$ for positives and negatives, usually with $\gamma_- > \gamma_+$ and often $\gamma_+ = 0$. It adds a probability margin $m$ that shifts a negative’s probability to $\max(p - m, 0)$, so very easy negatives are discarded entirely.

Long-tailed detection. On a long-tailed dataset such as LVIS, rare categories face a far more severe positive–negative imbalance than frequent ones, and one $\gamma$ for all categories does not suit them all. Equalized Focal Loss (Li et al., CVPR 2022) gives category $j$ its own focusing factor $\gamma_j = \gamma_b + s(1 - g_j)$. Here $g_j$ is the accumulated ratio of positive to negative gradients for that category, clamped to $[0, 1]$, and a weighting factor further raises rare categories. On the LVIS v1 val split with ResNet-50-FPN, a 2x schedule and repeat-factor sampling, EFL reached 27.5 AP against 25.7 for the authors’ improved one-stage baseline, and 20.2 against 14.3 on rare categories (their Table 2). The authors also report that most one-stage detectors trained on LVIS were unstable, with fluctuating results and sometimes NaN problems, which they attribute to abnormal gradients early in training. They stabilized them with a longer warm-up and gradient clipping.

Common Misconceptions

• “The focal loss is a class weight.” The class weight is $\alpha$, which the paper treats as a separate, optional factor. $\gamma$ weights examples by how well they are classified, regardless of class: an easy positive is down-weighted as much as an easy negative. In RetinaNet’s best setting, $\alpha = 0.25$ even favours the background.
• “α is the weight of the rare class, so it should be large.” Under cross-entropy the best $\alpha$ in Lin et al.’s ablation was 0.75. With $\gamma = 2$ it was 0.25. In CornerNet’s variant, $\alpha$ is the focusing exponent, not a weight at all.
• “The focal loss replaces sampling and initialization.” RetinaNet trains with the focal loss and the prior initialization of the classifier bias, and normalizes by the number of positive anchors. The RetinaNet article’s experiment shows how these choices interact.
• “Scores trained with the focal loss are probabilities.” For $\gamma > 0$ the loss is not strictly proper. Its exact minimizer is under-confident in the binary case, and with $\alpha$ even the decision threshold moves away from 0.5. Thresholds and calibration have to be set on validation data, or the scores mapped back with Charoenphakdee et al.’s transformation where it applies.
• “The focal loss always improves calibration.” Mukhoti et al. reported that it can give a lower ECE than cross-entropy, as Charoenphakdee et al. describe them. Charoenphakdee et al. found counter-examples, and suggest that such gains come from the loss’s under-confidence offsetting a network’s over-confidence.
• “The focal loss is a robust loss.” Lin et al. describe the opposite role. A robust loss down-weights the hard examples, the outliers. The focal loss down-weights the easy ones and concentrates training on the hard examples.

Where It Is Used

Dense object detection. After RetinaNet, the focal loss was adopted as the classification loss of many one-stage and anchor-free detectors. FCOS keeps RetinaNet’s focal loss (with $\alpha = 0.25$ and $\gamma = 2$ in its official code), EfficientDet keeps it with $\gamma = 1.5$, as the RetinaNet article reports, and CornerNet and CenterNet use the penalty-reduced variant for heatmaps. Detectors that predict localization quality moved to GFL or Varifocal loss. Which locations count as positives is a separate question, covered in label assignment, and the object detection and anchor-free object detection articles place these detectors in context.

Segmentation. Segment Anything (Kirillov et al., ICCV 2023) supervises its masks with a linear combination of the focal loss and the Dice loss (see Dice coefficient), the combination used in DETR. Per-pixel segmentation of small structures has the same imbalance between foreground and background pixels as dense detection, and Charoenphakdee et al. cite uses of the focal loss in medical imaging, among them brain-tumour and cardiac MRI segmentation.

Classification. Multi-label classification uses the focal loss and its asymmetric variant for the imbalance between present and absent labels. Single-label classifiers have used it as a training loss, and Mukhoti et al., as Charoenphakdee et al. summarize them, suggested that it gives more reliable confidence, with the caveats above.

Implementations. TorchVision provides torchvision.ops.sigmoid_focal_loss (as of November 2026, release 0.29.2), and RetinaNet’s original implementation is in Facebook’s Detectron, described in the RetinaNet article. Because the loss depends only on $p_t$, it is a few lines on top of a numerically stable binary cross-entropy. The common pitfalls lie elsewhere: the normalization by positives, the bias initialization, and choosing $\alpha$ and $\gamma$ together.