L1 and L2 Losses

The L2 loss and the L1 loss score a numerical prediction by the squared and the absolute difference from its target. They are the default losses for regression in vision: the offsets of a bounding box, a depth or disparity map, an optical flow field, the pixels of a restored image, the coordinates of a keypoint. The L2 loss is also called squared error or, averaged, the mean squared error (MSE); the L1 loss, absolute error or mean absolute error (MAE). Between them lies a family of robust losses that behave like L2 for small errors and like L1, or something gentler, for large ones: the Huber loss, the smooth L1 loss of Fast R-CNN, the Charbonnier loss and Barron’s general robust loss. The choice among them decides two things: which summary of an uncertain target the network learns to predict (a mean, a median or something in between), and how strongly a large error, often an outlier, pulls on the weights.

Definition

For a prediction $\hat{y}$ and target $y$ with residual $r = \hat{y} - y$, the L2 loss is $r^2$ (often written $\tfrac{1}{2}r^2$) and the L1 loss is $\lvert r \rvert$. For an output with many components, such as an image, the per-component losses are summed or averaged. A robust loss $\rho(r)$ is any loss that grows more slowly than $r^2$ for large residuals, so that large errors have a bounded or decreasing influence on the fit.

The names come from the $\ell_2$ and $\ell_1$ norms of the residual vector: the L2 loss is the squared $\ell_2$ norm and the L1 loss the $\ell_1$ norm. The squared and the unsquared $\ell_2$ norm differ: endpoint error for optical flow, for instance, averages the unsquared Euclidean length of each pixel’s 2D error vector, which treats the vector’s length like an absolute error.

Intuition

Suppose a network must predict one number for an input whose target varies: the depth behind a blurred patch, the position of an edge that could lie a few pixels left or right. Under L2, the best constant answer is the one that minimizes the average squared distance to all the possible targets, which is their mean; under L1, it is their median. The mean is pulled toward every possibility, including rare and extreme ones; the median sits with the majority and ignores how far the minority lies. In images, the mean of several plausible outputs is a blend of them, the familiar blur of squared-error regression, while the median takes, pixel by pixel, the middle value of the possibilities.

The gradients tell the same story from the training side. Under L2 the gradient is the residual itself, so a box that is off by ten units pulls ten times harder than one that is off by one; under L1 every error pulls with the same strength. The robust losses keep L2’s proportional pull near zero, where it gives smooth, stable convergence, and cap or reduce it for large errors.

Formal Definition

L2 and L1 as estimators

Training minimizes the average loss over the training images. With a flexible enough model, the minimizer of the expected squared error $\mathbb{E}\lVert \mathbf{y} - f(\mathbf{x}) \rVert^2$ is the conditional mean $f^*(\mathbf{x}) = \mathbb{E}[\mathbf{y} \mid \mathbf{x}]$, and the minimizer of the expected absolute error is the conditional median, component by component (Goodfellow, Bengio and Courville, 2016, §6.2.1.2, Eqs. 6.14–6.16). For a single number $u$ fitted to samples $y_1, \dots, y_n$ with weights $p_k$,

$$\arg\min_u \sum_k p_k (u - y_k)^2 = \sum_k p_k y_k, \qquad \arg\min_u \sum_k p_k \lvert u - y_k \rvert = \operatorname{median}_p(y_1, \dots, y_n),$$

where the weighted median may be any point of an interval when the weights split exactly in half. The median is therefore not always unique, while the mean always is.

Both losses are also negative log-likelihoods. If the target is the network’s output plus Gaussian noise of fixed variance, maximizing the likelihood minimizes the squared error (Goodfellow et al., 2016, §5.5.1); with Laplace noise of fixed scale it minimizes the absolute error. Barron (2019, §3.2) describes the absolute loss of a depth network in exactly these terms, as maximizing the likelihood of a Laplacian of fixed scale. The derivations are in Loss Function and Maximum Likelihood Estimation. The Laplace density has heavier tails than the Gaussian, so the L1 loss expects occasional large errors and is not surprised by them.

Huber and smooth L1

The Huber loss with threshold $\delta > 0$ is quadratic inside $[-\delta, \delta]$ and linear outside, joined so that its slope is continuous:

$$\rho_{\text{Huber}}(r) = \begin{cases} \tfrac{1}{2} r^2 & \lvert r \rvert < \delta, \\ \delta \big(\lvert r \rvert - \tfrac{1}{2}\delta\big) & \text{otherwise}. \end{cases}$$

It is named after Huber’s 1964 work on robust estimation of a location parameter (not read here); the form above is the one PyTorch implements as HuberLoss (PyTorch 2.14.1, torch/nn/modules/loss.py). The robust costs of nonlinear least squares use the same function with a threshold $k$.

The smooth L1 loss is the box-regression loss of Fast R-CNN (Girshick, 2015, Eq. 3) with the threshold fixed at 1: $0.5x^2$ for $\lvert x \rvert < 1$ and $\lvert x \rvert - 0.5$ otherwise, which is the Huber loss with $\delta = 1$. PyTorch’s SmoothL1Loss generalizes it with a threshold $\beta$, as $0.5 r^2/\beta$ inside and $\lvert r \rvert - 0.5\beta$ outside, and its documentation notes that this equals the Huber loss divided by $\beta$ (with $\delta = \beta$). The two differ in scale, not in shape: as $\beta \to 0$ smooth L1 tends to the L1 loss while the Huber loss tends to zero, and as $\delta \to \infty$ the Huber loss tends to the L2 loss while smooth L1 tends to zero. Because a loss divided by a constant has the same minimizer, Huber with $\delta$ and smooth L1 with $\beta = \delta$ fit the same function; they differ when they are added to other losses with fixed weights, or trained with a fixed learning rate.

Charbonnier

The Charbonnier loss is a differentiable approximation to the absolute value,

$$\rho_{\text{Charb}}(r) = \sqrt{r^2 + \varepsilon^2},$$

which behaves like $\varepsilon + r^2 / (2\varepsilon)$ for $\lvert r \rvert \ll \varepsilon$ and like $\lvert r \rvert$ for $\lvert r \rvert \gg \varepsilon$. LapSRN trained its super-resolution network with this penalty and $\varepsilon = 10^{-3}$, describing it as a differentiable variant of the $\ell_1$ norm (Lai et al., 2017, §3.2, Eq. 1). The function’s history in the literature differs by source: LapSRN credits it to Bruhn, Weickert and Schnörr (2005), a paper on optical flow, while Barron (2019) cites Charbonnier, Blanc-Féraud, Aubert and Barlaud (1994) and lists “pseudo-Huber” and “L1-L2” as other names for it (neither earlier paper was read here). Variational optical flow methods used the Charbonnier penalty before networks, for example Brox et al., as Brightness Constancy describes; the generalized Charbonnier loss $(r^2 + \varepsilon^2)^{\alpha/2}$ adds an exponent; Barron writes that it has been used in flow and depth estimation. PWC-Net’s fine-tuning loss, $(\lvert r \rvert + \epsilon)^q$ with $q < 1$, is a related robust form described in the PWC-Net article.

Barron’s general robust loss

Barron (2019) joined many of these losses into one function of the residual $x$, a shape parameter $\alpha$ and a scale $c > 0$:

$$\rho(x, \alpha, c) = \frac{\lvert \alpha - 2 \rvert}{\alpha} \left( \left( \frac{(x/c)^2}{\lvert \alpha - 2 \rvert} + 1 \right)^{\alpha/2} - 1 \right),$$

with its limits at $\alpha = 2$, $\alpha = 0$ and $\alpha \to -\infty$ defined separately (Eq. 8). The special cases are (Eqs. 2–7):

$\alpha$	Loss
2	L2, $\tfrac{1}{2}(x/c)^2$
1	Charbonnier (pseudo-Huber), $\sqrt{(x/c)^2 + 1} - 1$
0	Cauchy (Lorentzian), $\log\big(\tfrac{1}{2}(x/c)^2 + 1\big)$
−2	Geman–McClure, $2(x/c)^2 / \big((x/c)^2 + 4\big)$
$-\infty$	Welsch (Leclerc), $1 - \exp\big(-\tfrac{1}{2}(x/c)^2\big)$

The L1 loss itself is not a member, but $\alpha = 1$ with $c$ much smaller than the residuals approximates it up to scale and shift (Eq. 15). The $\alpha = 1$ member is the Charbonnier loss above with $\varepsilon = c$, divided by $c$ and shifted, so it has the same minimizer. Barron also reads $\alpha$ as a choice of average: minimizing the loss with $\alpha = 2$ estimates a mean, with $\alpha = 1$ something similar to a median, and with $\alpha \to -\infty$ a local mode (§1).

Properties

Gradients and outliers. The derivative $\psi(r) = \rho'(r)$, the influence of one residual on the fit, separates the family:

$$\psi_{L2}(r) = r, \quad \psi_{L1}(r) = \operatorname{sign}(r), \quad \psi_{\text{Huber}}(r) = \operatorname{clip}(r, -\delta, \delta), \quad \psi_{\text{smooth}}(r) = \operatorname{clip}(r/\beta, -1, 1), \quad \psi_{\text{Charb}}(r) = \frac{r}{\sqrt{r^2 + \varepsilon^2}}.$$

The L2 influence grows without bound; the L1, Huber, smooth L1 and Charbonnier influences are bounded, by 1, $\delta$, 1 and 1. For Barron’s loss with $\alpha < 1$ the influence rises and then falls again as the residual grows past $c$ (“redescending”, in M-estimation terms), so a large outlier has less effect than a moderate one, and as $\alpha \to -\infty$ residuals larger than about $3c$ are almost ignored (Barron, 2019, §1). Girshick (2015) gives the reason for smooth L1 in these terms: it is less sensitive to outliers than the L2 loss of R-CNN and SPPnet, which with unbounded targets can need careful learning-rate tuning to avoid exploding gradients. Bounding Box Regression prints the gradients of L2, smooth L1 and the balanced L1 loss of Libra R-CNN at several error sizes; Loss Function shows what the bounded influence does to a burst of frames corrupted by impulse noise.

Behaviour near zero. L1’s gradient does not shrink as the error does, so with a fixed step size gradient descent keeps overshooting the minimum instead of settling (our reasoning; the Loss Function example shrinks its step for this reason), and its derivative is undefined at zero. The quadratic zone of Huber, smooth L1 and Charbonnier removes the kink and makes the gradient vanish smoothly at the target. The width of that zone sets where “small” ends.

Convexity. L2, L1, Huber, smooth L1 and Charbonnier are convex in the residual, so for a fixed set of targets they have no spurious local minima. Barron’s loss is non-convex for $\alpha < 1$. Barron (2019) turns this into a schedule: start with a convex $\alpha$ and lower it during optimization, a form of graduated non-convexity (§1).

Scale. Every robust loss has a scale ($\delta$, $\beta$, $\varepsilon$, $c$) in the units of the residual. Barron’s loss is invariant to scaling the residual and $c$ together (§1, Eq. 11), so a threshold that is right for depth in metres is wrong for depth in millimetres, and Fast R-CNN normalizes its regression targets to zero mean and unit variance before applying a threshold of 1 (Girshick, 2015). L2 and L1 have no scale, but their weight against other losses changes with the units: L2 by the square of a unit change, L1 linearly.

Gradient size and learning rate. Under L2, the gradient shrinks with the error. LapSRN’s authors trained their L2 baseline with a larger learning rate because, they write, the gradient magnitude of the L2 loss is smaller, and found it needed more iterations to reach the accuracy of SRCNN (Lai et al., 2017, §4.1). Goodfellow et al. (2016, §6.2.1.2) add that L2 and L1 combined with saturating output units can give very small gradients, one reason classification uses cross-entropy.

Adapting the robustness. Lowering $\alpha$ never raises Barron’s loss (Eq. 12), so a network allowed to choose $\alpha$ by minimizing the loss would simply make it as small as possible. Barron instead minimizes the negative log-likelihood of a probability density built from the loss, whose normalizing constant penalizes small $\alpha$ for inliers, and lets each output dimension learn its own $\alpha$ (§2, §3). Applied to wavelet coefficients in an unsupervised monocular depth network on KITTI (Barron’s term; self-supervised in today’s usage), this adaptive loss reduced the paper’s average error measure, the geometric mean of the four standard depth errors and of one minus each of the three accuracies, by about 17% against the network’s original absolute-error loss; no single fixed $\alpha$ was best for all coefficients (§3.2).

Examples

What each loss predicts when the target position is uncertain. A network often cannot tell from its input exactly where a structure lies: an edge in an upsampled image, an object in a predicted future frame, a boundary in a depth map. The example models this directly. A $32 \times 112$ image with values 0 and 1 holds three vertical bars of widths 2, 4 and 6 pixels and a 24-pixel block, whose left edge is the step edge measured below. The true target is this image shifted horizontally by one of $-4, -2, 0, 2$ or $4$ pixels. Under two shift distributions, uniform and one in which the unshifted image has probability 0.6, the script computes, for every pixel, the exact output that minimizes the expected loss: by bisection on the derivative, since every loss here is convex. Before the results, it checks the data, that every minimizer recovers the image when there is only one possible target, that the L2 and L1 minimizers equal the weighted mean and median, that Huber with $\delta = 1$ equals L2 (every residual here is at most 1 in size) and smooth L1 with $\beta$ equals Huber with $\delta = \beta$, that no point on a fine grid has a lower expected loss, and that the derivatives match finite differences.

For each output, along a row through all four structures, the script reports the largest jump between neighbouring pixels at the edge (1 for a sharp step), the peak of each thin bar (1 if it survives intact, 0 if erased), the expected squared and absolute errors over the possible targets, and the mean squared error to the nearest of the five possible targets.

import numpy as np

# A 32x112 image with intensities 0 and 1: vertical bars 2, 4 and 6 pixels wide
# and a 24-pixel block, whose left edge is the step edge measured below.
H, W = 32, 112
base = np.zeros((H, W))
for c0, w in [(14, 2), (30, 4), (46, 6), (64, 24)]:
    base[4:28, c0:c0 + w] = 1.0

# The target is the image shifted horizontally by one of five offsets, with
# probabilities p: positional ambiguity, as when a network cannot tell from its
# input where exactly a structure lies.
shifts = [-4, -2, 0, 2, 4]
Y = np.stack([np.roll(base, s, axis=1) for s in shifts])
conditions = {"uniform": np.full(5, 0.2), "dominant": np.array([0.1, 0.1, 0.6, 0.1, 0.1])}

# Each loss rho(r) of the residual r = u - y, with its derivative psi(r).
def huber(d):
    return (lambda r: np.where(np.abs(r) < d, 0.5 * r**2, d * (np.abs(r) - 0.5 * d)),
            lambda r: np.clip(r, -d, d))
def smooth_l1(b):            # PyTorch's form: Huber divided by beta
    return (lambda r: np.where(np.abs(r) < b, 0.5 * r**2 / b, np.abs(r) - 0.5 * b),
            lambda r: np.clip(r / b, -1, 1))
def charbonnier(e):          # LapSRN's form
    return (lambda r: np.sqrt(r**2 + e**2), lambda r: r / np.sqrt(r**2 + e**2))

losses = {"L2": (lambda r: 0.5 * r**2, lambda r: r),
          "L1": (np.abs, np.sign)}
for d in [0.05, 0.2, 1.0]:
    losses[f"Huber {d}"] = huber(d)
for e in [0.001, 0.05, 0.2, 1.0]:
    losses[f"Charbonnier {e}"] = charbonnier(e)
checks_only = {f"smooth L1 {b}": smooth_l1(b) for b in [0.05, 0.2, 1.0]}

def minimize(psi, p):
    """Exact per-pixel minimizer of sum_k p_k rho(u - y_k) for a convex rho:
    bisection on the derivative over [0, 1], vectorized over pixels."""
    lo, hi = np.zeros((H, W)), np.ones((H, W))
    for _ in range(100):
        mid = (lo + hi) / 2
        g = np.tensordot(p, psi(mid - Y), axes=1)
        lo, hi = np.where(g < 0, mid, lo), np.where(g < 0, hi, mid)
    return (lo + hi) / 2

def expected(rho, u, p):
    return np.tensordot(p, rho(u - Y), axes=1)

# --- Checks -------------------------------------------------------------------
print("data: distinct targets", len({t.tobytes() for t in Y}),
      "| content within 10 px of the borders", Y[:, :, :10].sum() + Y[:, :, -10:].sum(),
      "| max targets covering a bar-2 pixel", int(Y[:, 16, 8:22].sum(0).max()))
row = base[16]
edges = np.flatnonzero(np.diff(np.r_[0, row, 0]))
print("data: bar widths in the base image", [int(w) for w in np.diff(edges)[::2]])

def wmedian(p):
    order = np.argsort(Y, axis=0, kind="stable")
    cum = np.cumsum(p[order], axis=0)
    idx = (cum >= 0.5 - 1e-12).argmax(axis=0)
    return np.take_along_axis(np.take_along_axis(Y, order, 0), idx[None], 0)[0]

single = np.array([0, 0, 1.0, 0, 0])
err = max(np.abs(minimize(f[1], single) - base).max() for f in {**losses, **checks_only}.values())
print(f"check C1, one possible target: max |minimizer - target| = {err:.1e}")

rr = np.array([-0.7, -0.3, -0.03, 0.02, 0.15, 0.6])   # residuals away from kinks
fd = max(np.max(np.abs((f[0](rr + 1e-7) - f[0](rr - 1e-7)) / 2e-7 - f[1](rr)) / np.abs(f[1](rr)))
         for f in {**losses, **checks_only}.values())
print(f"check C4, derivatives vs finite differences: max relative error {fd:.1e}")

grid = np.linspace(0, 1, 10001)
outputs = {}
for cname, p in conditions.items():
    out = {name: minimize(f[1], p) for name, f in losses.items()}
    outputs[cname] = out
    c2 = max(np.abs(out["L2"] - np.tensordot(p, Y, axes=1)).max(),
             np.abs(out["L1"] - wmedian(p)).max(),
             np.abs(out["Huber 1.0"] - out["L2"]).max(),
             *(np.abs(minimize(checks_only[f"smooth L1 {b}"][1], p) - out[f"Huber {b}"]).max()
               for b in [0.05, 0.2, 1.0]))
    # global optimality on a grid: only distinct pixel distributions need checking
    cols = {Y[:, i, j].tobytes(): (i, j) for i in range(H) for j in range(W)}
    c3 = -np.inf
    for name, (rho, _) in losses.items():
        for i, j in cols.values():
            y = Y[:, i, j]
            u = out[name][i, j]
            f_u = p @ rho(u - y)
            f_grid = (p[:, None] * rho(grid[None] - y[:, None])).sum(0).min()
            c3 = max(c3, f_u - f_grid)
    print(f"check C2 ({cname}): closed forms (mean, median, Huber 1 = L2, smooth L1 = Huber): "
          f"max diff {c2:.1e}")
    print(f"check C3 ({cname}): largest excess of solver over grid minimum {c3:.1e}")

# --- Results ------------------------------------------------------------------
def report(u, p):
    r = u[16]
    return (np.abs(np.diff(r[56:73])).max(), r[8:22].max(), r[24:40].max(), r[40:58].max(),
            np.mean(np.tensordot(p, (u - Y) ** 2, axes=1)),
            np.mean(np.tensordot(p, np.abs(u - Y), axes=1)),
            min(np.mean((u - t) ** 2) for t in Y))

for cname, p in conditions.items():
    print(f"\n{cname}: p = {p}")
    print(f"{'loss':17s} {'edge':>5s} {'bar2':>5s} {'bar4':>5s} {'bar6':>5s} "
          f"{'E[sq]':>7s} {'E[abs]':>7s} {'nearest':>7s}")
    for name in losses:
        e = report(outputs[cname][name], p)
        print(f"{name:17s} {e[0]:5.2f} {e[1]:5.2f} {e[2]:5.2f} {e[3]:5.2f} "
              f"{e[4]:7.4f} {e[5]:7.4f} {e[6]:7.4f}")

Output (NumPy 2.3.5; the script runs in under a second, and two runs gave identical output):

data: distinct targets 5 | content within 10 px of the borders 0.0 | max targets covering a bar-2 pixel 1
data: bar widths in the base image [2, 4, 6, 24]
check C1, one possible target: max |minimizer - target| = 3.9e-31
check C4, derivatives vs finite differences: max relative error 3.9e-08
check C2 (uniform): closed forms (mean, median, Huber 1 = L2, smooth L1 = Huber): max diff 5.6e-17
check C3 (uniform): largest excess of solver over grid minimum 1.4e-17
check C2 (dominant): closed forms (mean, median, Huber 1 = L2, smooth L1 = Huber): max diff 1.1e-16
check C3 (dominant): largest excess of solver over grid minimum 1.4e-17

uniform: p = [0.2 0.2 0.2 0.2 0.2]
loss               edge  bar2  bar4  bar6   E[sq]  E[abs] nearest
L2                 0.20  0.20  0.40  0.60  0.0696  0.1393  0.0482
L1                 1.00  0.00  0.00  1.00  0.1045  0.1045  0.0402
Huber 0.05         0.93  0.01  0.03  0.97  0.0992  0.1069  0.0383
Huber 0.2          0.73  0.05  0.13  0.87  0.0859  0.1143  0.0352
Huber 1.0          0.20  0.20  0.40  0.60  0.0696  0.1393  0.0482
Charbonnier 0.001  1.00  0.00  0.00  1.00  0.1043  0.1045  0.0401
Charbonnier 0.05   0.91  0.01  0.04  0.96  0.0977  0.1074  0.0378
Charbonnier 0.2    0.66  0.05  0.17  0.83  0.0827  0.1157  0.0352
Charbonnier 1.0    0.25  0.16  0.38  0.62  0.0700  0.1346  0.0450

dominant: p = [0.1 0.1 0.6 0.1 0.1]
loss               edge  bar2  bar4  bar6   E[sq]  E[abs] nearest
L2                 0.60  0.60  0.70  0.80  0.0469  0.0938  0.0121
L1                 1.00  1.00  1.00  1.00  0.0589  0.0589  0.0000
Huber 0.05         0.98  0.97  0.98  0.99  0.0574  0.0611  0.0001
Huber 0.2          0.90  0.87  0.91  0.95  0.0533  0.0675  0.0009
Huber 1.0          0.60  0.60  0.70  0.80  0.0469  0.0938  0.0121
Charbonnier 0.001  1.00  1.00  1.00  1.00  0.0589  0.0590  0.0000
Charbonnier 0.05   0.97  0.96  0.98  0.99  0.0572  0.0612  0.0001
Charbonnier 0.2    0.90  0.83  0.91  0.95  0.0530  0.0676  0.0011
Charbonnier 1.0    0.67  0.62  0.74  0.84  0.0472  0.0869  0.0086

The checks confirm that the solver finds the exact minimizers, so the table shows what each loss asks for, independent of any network or optimizer.

Under uniform uncertainty, L2 does what the mean must: the edge becomes a staircase of five steps of 0.2, spread over the eight pixels between the edge’s first and last possible positions, and each bar is smeared into a faint band whose peak is its width divided by 10, because the five copies lie 2 pixels apart. L1 behaves differently, and not simply as a sharper L2. The edge stays a perfect step, placed at the middle position. In our computation the reason is that a pixel takes the value 1 under L1 when at least three of the five shifted copies cover it, and on a structure wider than the spread of positions that majority boundary is itself a sharp edge. The same rule erases the 2- and 4-pixel bars completely, because no pixel of either is covered by three copies, while the 6-pixel bar is kept intact. So on this image the per-pixel median’s answer to positional uncertainty is to keep what most possibilities agree on and delete thin structures that do not line up, rather than to blur. Isola et al. (2017, §4.2) write that L1 encourages blur when it is uncertain where an edge lies; the computation here does not test their trained networks and shows only what the exact per-pixel L1 minimizer does on this image. Textures whose phase is unknown, rather than structures whose position is unknown, are the case shown in the image-to-image translation example.

The robust losses move between the two ends with their scale. With a threshold of 0.05 the Huber and Charbonnier minimizers are close to L1; at 0.2 the edge is partly softened and the thin bars come back faintly; Huber with $\delta = 1$ reproduces L2 exactly, because every residual in a 0–1 image lies inside its quadratic zone, and Charbonnier with $\varepsilon = 1$ comes close to L2 without reaching it. The scale is therefore a statement about which residuals count as “small” in the units of the data. The expected-error columns show that each end is optimal for its own criterion: L2 has the lowest expected squared error, L1 the lowest expected absolute error. Under uniform uncertainty, no output is a plausible image, and the intermediate scales (Huber and Charbonnier at 0.2) end up closest to one of the five possible targets: in our reading, they keep much of L1’s edge contrast while leaving faint traces of the bars that L1 deletes.

When one position has probability 0.6, more than half, the weighted median at every pixel is that position’s value, so L1 returns the most likely image exactly (nearest error 0) and the small-scale robust losses nearly so. L2 still blends in the other four positions with weight 0.4.

Common Misconceptions

• “The L1 loss is L1 regularization.” The L1 and L2 losses penalize the residual between output and target. L1 and L2 regularization penalize the weights, the latter usually as weight decay. The two are independent choices, and both can appear in one objective; see Regularization.
• “L1 gives sharp images and L2 blurry ones.” Both summarize uncertain targets. In the example, the L1 minimizer keeps an isolated edge sharp but deletes thin structures whose position is uncertain, and Isola et al. (2017, §3.2.2) note that L1 also produces blurry results in image generation. In the example, the L1 output is sharp only where a majority of the plausible targets agree.
• “Huber and smooth L1 are different losses.” They are the same function up to a factor of $\delta$ (PyTorch 2.14.1 documentation), and Fast R-CNN’s smooth L1 is Huber with $\delta = 1$. The factor still matters: smooth L1’s gradient is capped at 1 for every $\beta$, Huber’s at $\delta$, so raising the threshold raises the pull of large errors under Huber but not under smooth L1, and the two shift the balance against a classification loss differently.
• “MSE loss” and “L2 loss” always mean the same number. Conventions differ: $\tfrac{1}{2}r^2$ or $r^2$, a sum or a mean over pixels, and the squared or unsquared Euclidean norm for vector outputs. They change the gradient’s scale, and with it the effective learning rate and the balance against other losses. Mean Squared Error covers the conventions as evaluation metrics.
• “A robust loss makes a network ignore bad labels.” Bounded losses (L1, Huber, smooth L1, Charbonnier) limit an outlier’s pull but never ignore it; only redescending losses such as Barron’s with $\alpha < 1$ reduce the influence of very large residuals, and they are not convex.

Where It Is Used

• Box regression. Fast R-CNN used smooth L1 on normalized box offsets (Girshick, 2015), and Faster R-CNN and many one-stage detectors kept it; DETR combines an L1 loss on normalized box coordinates with a GIoU loss. Bounding Box Regression covers these, balanced L1 and the losses built on overlap.
• Keypoints. Coordinate regression of facial landmarks compares L2, L1 and smooth L1 losses and the Wing loss, which is logarithmic for small errors and linear for large ones, as Facial Landmark Detection describes; heatmap methods instead regress a map with a per-pixel loss (heatmap regression).
• Optical flow. FlowNet trained on the endpoint error, RAFT on an L1 distance over its iterative predictions (see Loss Function and RAFT), and PWC-Net fine-tunes with a robust power of the L1 norm. Classical variational methods used robust penalties such as the Charbonnier penalty (see Brightness Constancy).
• Depth. Supervised depth estimation regresses depth maps with per-pixel losses; self-supervised methods score a reconstructed image with photometric losses such as L1 or a mix of L1 and SSIM terms (self-supervised depth estimation), and Barron (2019) replaced the absolute RGB difference used by one such network, SfMLearner, with his adaptive loss.
• Image restoration. Super-resolution and image denoising networks were long trained with L2 (for super-resolution, Lai et al., 2017, Table 1), which maximizes PSNR (Lim et al., 2017, §4.2). LapSRN used the Charbonnier loss and reported that a version trained with L2 showed more ringing artifacts (Lai et al., 2017, §4.1, Fig. 3). EDSR switched from L2 to L1 because the authors found it converged better (Lim et al., 2017, §4.2). In their Table 2 (§4.4), SRResNet trained with L1 scored slightly higher than with L2 on the DIV2K validation set at ×2 (34.44 against 34.40 dB PSNR) and ×3 (30.85 against 30.82 dB), and the same at ×4 (28.92 dB, SSIM 0.8961 against 0.8960); the text describes L1 as slightly better for all scale factors.
• Image generation and translation. Pixel losses alone produce averages of the plausible outputs: Isola et al. (2017, §1) explain the blur of Euclidean losses this way, and pix2pix adds a pixel term to an adversarial loss, choosing L1 over L2 because L1 encourages less blurring (§3.1), and lets L1 enforce the low frequencies while the discriminator models the high ones (§3.2.2); see pix2pix. Perceptual losses compare features instead of pixels, and video prediction and variational autoencoders meet the same blur from squared error.
• Classical estimation. The same functions are the M-estimators of robust fitting in nonlinear least squares and bundle adjustment, where Huber and Cauchy losses limit the influence of mismatches.