Bounding Box Regression

Bounding box regression is the part of an object detector that turns a rough candidate, such as a region proposal, an anchor box or a point on a feature map, into the four numbers of a bounding box. It is also called box regression, bbox regression or localization, and its loss the localization or regression loss. Two design decisions define it: how the box is parameterized, that is, what the network actually outputs, and which loss compares the prediction with the ground truth. This article covers both, from the offsets and log scales of R-CNN through smooth $L_1$ and balanced $L_1$ to the losses built on intersection over union (IoU, GIoU, DIoU, CIoU and others) and to detectors that predict a distribution over box positions. The IoU article covers the metric; this one covers its use as a training objective.

Definition

Bounding box regression is the supervised learning of a mapping from image features at a reference location to the coordinates of the box of the object assigned to that location. The reference may be a region proposal (R-CNN, Fast R-CNN), an anchor box (Faster R-CNN, SSD), a grid cell (YOLO), a feature-map location (FCOS), a corner keypoint (CornerNet) or a learned query (DETR). Which ground-truth box each reference must regress is decided beforehand by label assignment; the regression loss is computed only for references assigned to an object.

Intuition

A detector’s candidate is usually close to an object but not on it. Regression learns the correction: shift the box a little right, make it taller. Two things make this harder than ordinary regression. First, objects range from a few pixels to the whole image, so an error of 10 pixels is negligible for a bus and fatal for a distant pedestrian; the outputs must be expressed in a scale-free way. Second, detection is scored by IoU, not by coordinate error. Rezatofighi et al. (2019) show pairs of boxes whose corners are the same $\ell_2$ distance from the ground truth but whose IoU ranges from 0.26 to 0.65, so minimizing a coordinate loss is only loosely tied to the score that matters. The history of box regression is largely a sequence of attempts to fix one of these two problems without breaking the other.

Formal Definition

Parameterizations

Offsets and log scales. R-CNN (Girshick et al., 2014) added a regression stage, inspired by the box regression in deformable part models, that predicts a new detection window from the CNN features of a selective search proposal; the CVPR paper reports that it fixed many mislocalized detections and raised mAP by 3 to 4 points. With a reference box $a = (x_a, y_a, w_a, h_a)$ (centre, width, height) and a ground-truth box $(x^*, y^*, w^*, h^*)$, the targets are

$$t^*_x = \frac{x^* - x_a}{w_a}, \qquad t^*_y = \frac{y^* - y_a}{h_a}, \qquad t^*_w = \log\frac{w^*}{w_a}, \qquad t^*_h = \log\frac{h^*}{h_a},$$

as written in the Faster R-CNN paper (Ren et al., 2015), where the reference is an anchor rather than a proposal; Girshick (2015) describes them as a scale-invariant translation and a log-space shift of width and height. Decoding inverts the map, $w = w_a e^{t_w}$ and so on, so a predicted width is always positive. The R-CNN article derives its ridge-regression fit and the anchor boxes article the decoding. Faster R-CNN learns $k$ such regressors per position, one for each anchor shape, that share no weights.

Bounded offsets. The first YOLO (Redmon et al., 2016) regressed boxes directly with a squared error and predicted the square roots of width and height, so that the same deviation costs less in a large box than in a small one. YOLO9000 (Redmon and Farhadi, 2017) found the unconstrained anchor offsets of region proposal networks unstable early in training, since any anchor could be moved anywhere in the image, and instead predicted the centre relative to its grid cell through a logistic function, $b_x = \sigma(t_x) + c_x$, while keeping $b_w = p_w e^{t_w}$ for the size relative to the prior $p_w$.

Distances to the sides. FCOS (Tian et al., 2019) regresses, at a feature-map location $(x, y)$ inside a ground-truth box $(x_0, y_0, x_1, y_1)$, the four distances

$$l^* = x - x_0, \qquad t^* = y - y_0, \qquad r^* = x_1 - x, \qquad b^* = y_1 - y,$$

with $\exp(s_i x)$ on the output to keep them positive, where $s_i$ is a trainable scalar for pyramid level $i$. These distances are in pixels and grow with the object. Instead of assigning anchors of different sizes to different levels, FCOS limits the range each pyramid level must regress: a location whose largest distance falls outside the level’s interval is treated as negative. Anchor-free detection builds on this parameterization.

Corners. He et al. (2019) regress the four box edges separately, as offsets of each edge from the anchor’s edge divided by the anchor’s width or height, so that each coordinate can carry its own uncertainty (see below). CornerNet (Law and Deng, 2018) does not regress from a reference box at all: it detects the top-left and bottom-right corners as keypoints in heatmaps and regresses only a sub-pixel offset $o_k = (x_k/n - \lfloor x_k/n \rfloor, y_k/n - \lfloor y_k/n \rfloor)$ that recovers the precision lost by the network’s downsampling factor $n$, trained with smooth $L_1$ at the ground-truth corners.

Per-coordinate losses

R-CNN fitted its regressors with a squared ($L_2$) loss. Fast R-CNN (Girshick, 2015) replaced it with

$$\operatorname{smooth}_{L_1}(z) = \begin{cases} 0.5\, z^2 & \text{if } |z| < 1, \\ |z| - 0.5 & \text{otherwise,} \end{cases} \qquad L_{\text{loc}}(t, t^*) = \sum_{i \in \{x, y, w, h\}} \operatorname{smooth}_{L_1}(t_i - t^*_i),$$

a robust loss that, Girshick writes, is less sensitive to outliers than $L_2$: with unbounded targets, $L_2$ can require careful learning-rate tuning to avoid exploding gradients. The targets are normalized to zero mean and unit variance. Its gradient is $z$ for small errors and $\pm 1$ beyond, so no single box can pull harder than a fixed amount. With threshold 1 this is the Huber loss; Tychsen-Smith and Petersson (2018) write the general form $\tfrac{1}{2}z^2$ for $|z| < \tau$ and $\tau|z| - \tfrac{1}{2}\tau^2$ otherwise and call it the Huber loss, “also known as smooth L1 loss”. Faster R-CNN and SSD use smooth $L_1$ on offsets of the parameterization above, as do many one-stage object detectors.

Pang et al. (2019) observed in Libra R-CNN that, under smooth $L_1$, samples with a loss of 1 or more (outliers in their terms) dominate the gradient, while inliers, the accurately localized boxes, contribute on average only 30% of an outlier’s gradient per sample; raising the weight of the localization loss to compensate also amplifies the outliers. Their balanced $L_1$ loss keeps the gradient of outliers constant and raises that of inliers by defining the gradient first,

$$\frac{\partial L_b}{\partial z} = \begin{cases} \alpha \ln(b|z| + 1) & \text{if } |z| < 1, \\ \gamma & \text{otherwise,} \end{cases} \qquad L_b(z) = \begin{cases} \dfrac{\alpha}{b}(b|z| + 1)\ln(b|z| + 1) - \alpha|z| & \text{if } |z| < 1, \\ \gamma|z| + C & \text{otherwise,} \end{cases}$$

with $\alpha \ln(b + 1) = \gamma$ so that the gradient is continuous, and defaults $\alpha = 0.5$, $\gamma = 1.5$ (the paper writes the gradient through $|z|$, as a magnitude; as a function of $z$ it is odd). In the paper’s ablation (ResNet-50-FPN Faster R-CNN, COCO val-2017, 1× schedule), balanced $L_1$ alone raised AP from 35.9 to 37.2, while simply weighting smooth $L_1$ by 1.5 gave 36.4 and plain $L_1$ at best 36.6; inside the full Libra R-CNN it added 0.8 AP (37.7 to 38.5), mostly at AP75.

IoU-based losses

A loss on the four parameters treats them as independent, while IoU couples them, and an $\ell_n$ loss on pixel coordinates is not invariant to the object’s scale. Computing the IoU of two axis-aligned boxes needs only $\min$, $\max$ and products, all differentiable almost everywhere, so IoU can be optimized directly.

IoU loss. UnitBox (Yu et al., 2016) introduced an IoU-based box loss; the α-IoU paper (He et al., 2021) attributes to it the form $-\log(\mathrm{IoU})$, and later work mostly uses

$$L_{\mathrm{IoU}} = 1 - \mathrm{IoU}(B, B^{gt}) \in [0, 1],$$

as defined by Rezatofighi et al. (2019). FCOS uses the IoU loss “as in UnitBox” on its side distances. The loss is invariant to scale, but if the boxes do not overlap, IoU is 0 whatever the distance between them, and the gradient vanishes.

GIoU. Rezatofighi et al. (2019) subtract from IoU the fraction of the smallest enclosing box $C$ that neither box covers:

$$\mathrm{GIoU} = \mathrm{IoU} - \frac{|C \setminus (B \cup B^{gt})|}{|C|}, \qquad L_{\mathrm{GIoU}} = 1 - \mathrm{GIoU} \in [0, 2].$$

For disjoint boxes the loss reduces to $2 - |B \cup B^{gt}|/|C|$, which, the authors note, falls as the enclosing box shrinks and the predicted box grows, so the prediction moves toward overlap. Their Algorithm 2 also reorders the predicted corners so that $x_2 > x_1$ and $y_2 > y_1$, which keeps the loss defined and bounded for any network output. Replacing the default box loss (smooth $L_1$ in the final refinement stage of Faster R-CNN with a ResNet-50 backbone, weighted by 10) raised AP on PASCAL VOC 2007 test from 0.370 to 0.392 (0.384 with the plain IoU loss), and on the MS COCO 2018 validation set from 0.360 to 0.369 (0.368); for YOLO v3 on the COCO 2018 test set it went from 0.314 with its MSE loss to 0.333 (0.321). The authors attribute the smaller COCO gains with Faster R-CNN partly to its denser anchors, which make non-overlapping cases, where GIoU differs most from IoU, rarer. DETR combines the GIoU loss with an $L_1$ loss on normalized box coordinates.

DIoU and CIoU. Zheng et al. (2020) ran a simulation of 1,715,000 regression cases, in which anchor boxes of various positions, scales and aspect ratios were moved by gradient descent onto target boxes of seven aspect ratios. GIoU, they found, first enlarges the predicted box until it overlaps the target, then converges slowly, especially for boxes displaced horizontally or vertically; and when one box contains the other, $C$ equals the larger box, the penalty vanishes and GIoU reduces to IoU. They proposed adding penalties to $1 - \mathrm{IoU}$ instead:

$$L_{\mathrm{DIoU}} = 1 - \mathrm{IoU} + \frac{\rho^2(\mathbf{b}, \mathbf{b}^{gt})}{c^2}, \qquad L_{\mathrm{CIoU}} = L_{\mathrm{DIoU}} + \alpha v,$$ $$v = \frac{4}{\pi^2}\left(\arctan\frac{w^{gt}}{h^{gt}} - \arctan\frac{w}{h}\right)^2, \qquad \alpha = \frac{v}{(1 - \mathrm{IoU}) + v},$$

where $\rho$ is the Euclidean distance between the box centres $\mathbf{b}$ and $\mathbf{b}^{gt}$ and $c$ the diagonal of the smallest enclosing box. DIoU (Distance-IoU) penalizes centre distance; CIoU (Complete IoU) adds the consistency of aspect ratios, with $\alpha$ giving the overlap term priority when IoU is low. The paper gives the gradient of $v$ with respect to $w$ and $h$, which contains a factor $1/(w^2 + h^2)$; because that factor is large when $w$ and $h$ lie in $[0, 1]$, their implementation drops it while keeping the direction. The official simulation code also treats $\alpha$ as a constant when differentiating (Zheng et al., repository, dCIOU.m). With YOLO v3 (Darknet-608) trained on VOC 07+12 and tested on VOC 2007 test, AP over IoU thresholds 0.50 to 0.95 was 46.57 with the IoU loss, 47.73 with GIoU, 48.10 with DIoU and 49.21 with CIoU; with Faster R-CNN (ResNet-50-FPN) on COCO val2017 it was 37.93, 38.02, 38.09 and 38.65, though CIoU was slightly below the IoU loss on small objects (AP 21.32 against 21.58). The paper describes its gains as relative percentages, such as 5.67% for CIoU over the IoU loss with YOLO v3. The same paper uses DIoU as the overlap criterion in non-maximum suppression, so that boxes with distant centres are less likely to suppress each other.

Other variants. Tychsen-Smith and Petersson (2018) proposed the Bounded IoU loss, which applies a Huber loss to $1 - \mathrm{IoU}_B$, an upper bound on IoU obtained by varying one parameter at a time with the others set to the ground truth: $\mathrm{IoU}_B(x) = \max\big(0, (w^{gt} - 2|\Delta x|)/(w^{gt} + 2|\Delta x|)\big)$ for position and $\min(w/w^{gt}, w^{gt}/w)$ for width. Comparing it with R-CNN’s cost over the range $\mathrm{IoU} \ge 0.5$, they concluded that R-CNN’s loss puts too little weight on position. The α-IoU family (He et al., 2021) raises IoU and the penalty term to a power, $L = 1 - \mathrm{IoU}^\alpha + P^\alpha$, which recovers the IoU, GIoU, DIoU and CIoU losses at $\alpha = 1$; without the penalty, its Box-Cox form $(1 - \mathrm{IoU}^\alpha)/\alpha$ tends to $-\log(\mathrm{IoU})$ as $\alpha \to 0$; for $\alpha > 1$ it up-weights well-localized boxes, and the authors recommend $\alpha = 3$, or tuning $\alpha \in [2, 3]$. Many further penalty terms have been proposed since; they follow the same pattern of $1 - \mathrm{IoU}$ plus a geometric penalty.

Distribution-based prediction

The losses above treat each box edge as a single number, a Dirac delta in the language of Li et al. (2020). Ground-truth edges are often ambiguous, through occlusion, blur or inconsistent labelling, and two lines of work predict a distribution instead.

Gaussian edges. He et al. (2019) predict for each edge coordinate a mean $x_e$ and a standard deviation $\sigma$, model the ground truth as a Dirac delta at $x_g$, and minimize the KL divergence between the two, which up to constants is

$$L_{\text{reg}} \propto \frac{(x_g - x_e)^2}{2\sigma^2} + \frac{1}{2}\log \sigma^2 .$$

The network can lower the loss on ambiguous edges by predicting a large $\sigma$, which also scales down their gradient. In practice it predicts $\log \sigma^2$ for stability and switches to a smooth-$L_1$-like form when the error exceeds 1. At test time the variances weight neighbouring boxes in a “variance voting” step inside NMS. With VGG-16 Faster R-CNN on COCO (trained on train2014, tested on val2014), the KL loss alone raised AP from 23.6 to 26.4, and with soft-NMS and variance voting to 29.1; with ResNet-50-FPN Mask R-CNN on val2017, AP rose from 38.6 to 39.5 with the KL loss and to 40.4 with all three, and AP at IoU 0.9 from 11.3 to 17.5.

General distributions. Generalized Focal Loss (Li et al., 2020) makes no shape assumption. Each side distance $y$, in the range $[y_0, y_n]$, is discretized into bins $y_0, y_1, \dots, y_n$ with spacing $\Delta$ (1 in their experiments); a softmax over $n + 1$ outputs gives probabilities $S_i$, and the prediction is the expectation $\hat{y} = \sum_i S_i y_i$, which can still be trained with an IoU or GIoU loss. Because infinitely many distributions have the same mean, the distribution focal loss concentrates mass on the two bins around the target, $y_i \le y \le y_{i+1}$:

$$\mathrm{DFL}(S_i, S_{i+1}) = -\big((y_{i+1} - y)\log S_i + (y - y_i)\log S_{i+1}\big),$$

whose minimum, $S_i = (y_{i+1} - y)/(y_{i+1} - y_i)$, makes $\hat{y} = y$ exactly. In their training loss (Eq. 11), DFL with weight 1/4 is added to the GIoU loss with weight 2; the accompanying text calls the two weights those of QFL, their classification loss, and DFL, while the equation applies the first to the GIoU term. In the paper’s ablation on COCO minival (ResNet-50, 1× schedule), the general distribution with DFL gave 39.0 AP on FCOS and 39.5 on ATSS, against 38.5 and 39.2 with the usual single-value regression and 38.6 and 39.3 with a Gaussian. A flat predicted distribution signals an uncertain edge.

Properties

• Scale invariance. Offsets divided by the reference size, log size ratios and every IoU-based loss are unchanged when the image and boxes are scaled together; losses on pixel coordinates and FCOS’s raw side distances are not, which is one reason FCOS bounds the range per pyramid level. The first example below checks this on a synthetic image.
• Gradients where boxes do not overlap. The IoU loss is flat for disjoint boxes; GIoU, DIoU and CIoU are not, and neither are the coordinate losses. The coordinate losses’ pull does not fade with distance, while the IoU-based penalties are normalized by the enclosing box and weaken as it grows (second example).
• Nested boxes. When one box lies inside the other, IoU depends only on the ratio of their areas, so it follows that neither IoU nor GIoU (whose enclosing box is then the larger box) gives the inner box any push toward the centre; DIoU’s centre-distance term does.
• Bounded and unbounded losses. IoU-based losses are bounded (0 to 1 for IoU, 0 to 2 for GIoU), smooth $L_1$ and $L_2$ are not. Replacing one with the other changes the balance with the classification loss, and each paper retuned a weight: 10 for GIoU in Faster and Mask R-CNN, 5 for the IoU-based losses in SSD and 12 in Faster R-CNN in the DIoU paper, and 0.125 for the Bounded IoU loss in DeNet. Reported gains are therefore comparisons of loss and weight together.
• Positives only. The box loss is summed over the references assigned to objects; Fast R-CNN switches it off for background RoIs with an indicator $[u \ge 1]$. Which references count as positive, and how many, changes what the regressor is asked to learn as much as the loss does.

Examples

Parameterizations and per-coordinate gradients

The first script renders one elliptical object on a $160 \times 120$ image and the same scene at twice the resolution, reads the ground-truth box from each mask, and expresses it relative to a square anchor (scaled with the image) in three parameterizations. It then compares three losses between the anchor and the ground truth and prints the gradient of one coordinate’s error under $L_2$, smooth $L_1$ and balanced $L_1$ with Libra R-CNN’s defaults. It uses NumPy 2.3.5.

import numpy as np

def ellipse_mask(H, W, cx, cy, a, b):
    yy, xx = np.mgrid[0:H, 0:W] + 0.5            # pixel (i, j) covers [j, j+1) x [i, i+1)
    return ((xx - cx) / a) ** 2 + ((yy - cy) / b) ** 2 <= 1.0

def box_from_mask(m):                             # corners (x1, y1, x2, y2), continuous
    r, c = np.nonzero(m)
    return np.array([c.min(), r.min(), c.max() + 1, r.max() + 1], float)

def to_cxcywh(b):
    return np.array([(b[0] + b[2]) / 2, (b[1] + b[3]) / 2, b[2] - b[0], b[3] - b[1]])

def to_xyxy(c):
    return np.array([c[0] - c[2] / 2, c[1] - c[3] / 2, c[0] + c[2] / 2, c[1] + c[3] / 2])

def encode(gt, anchor):                           # R-CNN / Faster R-CNN targets
    g, a = to_cxcywh(gt), to_cxcywh(anchor)
    return np.array([(g[0] - a[0]) / a[2], (g[1] - a[1]) / a[3],
                     np.log(g[2] / a[2]), np.log(g[3] / a[3])])

def decode(t, anchor):
    a = to_cxcywh(anchor)
    return to_xyxy([a[0] + t[0] * a[2], a[1] + t[1] * a[3], a[2] * np.exp(t[2]), a[3] * np.exp(t[3])])

def sides(gt, px, py):                            # FCOS-style distances (l, t, r, b)
    return np.array([px - gt[0], py - gt[1], gt[2] - px, gt[3] - py])

def smooth_l1(x, beta=1.0):
    ax = np.abs(x)
    return np.where(ax < beta, 0.5 * ax ** 2 / beta, ax - 0.5 * beta)

def iou(p, g):
    iw = max(0.0, min(p[2], g[2]) - max(p[0], g[0]))
    ih = max(0.0, min(p[3], g[3]) - max(p[1], g[1]))
    inter = iw * ih
    union = (p[2] - p[0]) * (p[3] - p[1]) + (g[2] - g[0]) * (g[3] - g[1]) - inter
    return inter / union

# Part A: the same scene at two resolutions.
for s in (1, 2):
    img = ellipse_mask(120 * s, 160 * s, cx=70 * s, cy=55 * s, a=30 * s, b=18 * s)
    gt = box_from_mask(img)
    anchor = to_xyxy(np.array([78.0, 60.0, 48.0, 48.0]) * s)      # a square anchor near the object
    t = encode(gt, anchor)
    print(f"scale {s}: GT box {gt}")
    print("  R-CNN targets (tx, ty, tw, th)", np.round(t, 4))
    print("  sides (l, t, r, b) from point  ", sides(gt, 75 * s, 50 * s))
    print("  corner offsets GT - anchor      ", gt - anchor)
    print(f"  L1 on corners {np.abs(gt - anchor).sum():6.2f}   smooth L1 on targets "
          f"{smooth_l1(t).sum():.4f}   1 - IoU {1 - iou(anchor, gt):.4f}")
    print("  decode(encode(GT)) == GT:", np.allclose(decode(t, anchor), gt))

# Part B: gradient magnitude of one coordinate error x for three per-coordinate losses.
alpha, gamma = 0.5, 1.5
b = np.exp(gamma / alpha) - 1                     # from alpha * ln(b + 1) = gamma
def grad_l2(x):        return np.abs(x)
def grad_smooth_l1(x): return np.minimum(np.abs(x), 1.0)
def grad_balanced(x):  return np.where(np.abs(x) < 1, alpha * np.log(b * np.abs(x) + 1), gamma)
print(f"\nbalanced L1: alpha={alpha}, gamma={gamma}, b={b:.3f}")
print("  |x|     L2   smoothL1  balancedL1")
for x in [0.05, 0.1, 0.3, 0.6, 1.0, 2.0, 5.0]:
    print(f"{x:5.2f} {grad_l2(x):6.2f} {grad_smooth_l1(x):9.2f} {float(grad_balanced(x)):11.3f}")

Output:

scale 1: GT box [ 40.  37. 100.  73.]
  R-CNN targets (tx, ty, tw, th) [-0.1667 -0.1042  0.2231 -0.2877]
  sides (l, t, r, b) from point   [35. 13. 25. 23.]
  corner offsets GT - anchor       [-14.   1.  -2. -11.]
  L1 on corners  28.00   smooth L1 on targets 0.0856   1 - IoU 0.4103
  decode(encode(GT)) == GT: True
scale 2: GT box [ 80.  74. 200. 146.]
  R-CNN targets (tx, ty, tw, th) [-0.1667 -0.1042  0.2231 -0.2877]
  sides (l, t, r, b) from point   [70. 26. 50. 46.]
  corner offsets GT - anchor       [-28.   2.  -4. -22.]
  L1 on corners  56.00   smooth L1 on targets 0.0856   1 - IoU 0.4103
  decode(encode(GT)) == GT: True

balanced L1: alpha=0.5, gamma=1.5, b=19.086
  |x|     L2   smoothL1  balancedL1
 0.05   0.05      0.05       0.335
 0.10   0.10      0.10       0.534
 0.30   0.30      0.30       0.953
 0.60   0.60      0.60       1.261
 1.00   1.00      1.00       1.500
 2.00   2.00      1.00       1.500
 5.00   5.00      1.00       1.500

Doubling the resolution leaves the R-CNN targets, the smooth $L_1$ loss on them and the IoU loss unchanged, while the side distances, the corner offsets and the $L_1$ loss on corners double (the rendered mask scales exactly, so the boxes double without rounding). A loss on pixel coordinates therefore weights the same object twice as heavily at twice the resolution, four times if it is squared, and in a batch it weights large objects over small ones. The anchor’s IoU with the object is only 0.59, yet all four normalized targets are below 0.3 in magnitude, inside smooth $L_1$’s quadratic zone, where its gradient equals that of $L_2$. That is the regime Libra R-CNN targets: balanced $L_1$ gives a coordinate error of 0.1 a gradient of 0.53 instead of 0.1, while still capping every error above 1 at $\gamma = 1.5$.

Gradients for disjoint and nested boxes

The second script draws an elliptical object on a $160 \times 120$ image, reads its box, and evaluates five losses for four predicted boxes: the same size as the ground truth but displaced 6 pixels down and to the right with a gap of 10 or 60 pixels; a half-size box inside the ground truth, off-centre; and a concentric box inside it with a different aspect ratio. The losses do not depend on the image bounds, so the far box may extend past the image. Gradients with respect to the predicted centre, width and height (in pixels) are central finite differences. Smooth $L_1$ is applied to the R-CNN targets with the anchor equal to the current prediction, as when a head outputs zero offsets. Because the CIoU paper leaves open whether $\alpha$ is differentiated, CIoU is printed both ways.

import numpy as np

H, W = 120, 160
yy, xx = np.mgrid[0:H, 0:W] + 0.5
obj = ((xx - 60) / 30) ** 2 + ((yy - 60) / 18) ** 2 <= 1.0     # synthetic object
r, c = np.nonzero(obj)
gt = np.array([c.min(), r.min(), c.max() + 1, r.max() + 1], float)  # GT box (x1, y1, x2, y2)

def xyxy(p):                                   # p = (cx, cy, w, h)
    return np.array([p[0] - p[2] / 2, p[1] - p[3] / 2, p[0] + p[2] / 2, p[1] + p[3] / 2])

def cxcywh(b):
    return np.array([(b[0] + b[2]) / 2, (b[1] + b[3]) / 2, b[2] - b[0], b[3] - b[1]])

def parts(p):
    a, g = xyxy(p), gt
    iw = max(0.0, min(a[2], g[2]) - max(a[0], g[0]))
    ih = max(0.0, min(a[3], g[3]) - max(a[1], g[1]))
    inter = iw * ih
    union = (a[2] - a[0]) * (a[3] - a[1]) + (g[2] - g[0]) * (g[3] - g[1]) - inter
    cw = max(a[2], g[2]) - min(a[0], g[0])     # smallest enclosing box
    ch = max(a[3], g[3]) - min(a[1], g[1])
    gc = cxcywh(g)
    rho2 = (p[0] - gc[0]) ** 2 + (p[1] - gc[1]) ** 2
    v = 4 / np.pi ** 2 * (np.arctan(gc[2] / gc[3]) - np.arctan(p[2] / p[3])) ** 2
    return inter / union, union, cw * ch, rho2, cw ** 2 + ch ** 2, v

def l_iou(p):
    return 1 - parts(p)[0]

def l_giou(p):
    iou, union, area_c, *_ = parts(p)
    return 1 - iou + (area_c - union) / area_c

def l_diou(p):
    iou, _, _, rho2, c2, _ = parts(p)
    return 1 - iou + rho2 / c2

def l_ciou(p, alpha=None):                     # alpha=None: alpha recomputed from p (differentiated)
    iou, _, _, rho2, c2, v = parts(p)
    a = v / ((1 - iou) + v) if alpha is None else alpha
    return 1 - iou + rho2 / c2 + a * v

def make_smooth_l1(anchor):                    # smooth L1 on R-CNN targets relative to a fixed anchor
    gc = cxcywh(gt)
    tstar = np.array([(gc[0] - anchor[0]) / anchor[2], (gc[1] - anchor[1]) / anchor[3],
                      np.log(gc[2] / anchor[2]), np.log(gc[3] / anchor[3])])
    def loss(p):
        t = np.array([(p[0] - anchor[0]) / anchor[2], (p[1] - anchor[1]) / anchor[3],
                      np.log(p[2] / anchor[2]), np.log(p[3] / anchor[3])])
        d = np.abs(t - tstar)
        return np.where(d < 1, 0.5 * d ** 2, d - 0.5).sum()
    return loss

def grad(f, p, h=1e-4):                        # central differences w.r.t. (cx, cy, w, h)
    g = np.zeros(4)
    for k in range(4):
        e = np.zeros(4); e[k] = h
        g[k] = (f(p + e) - f(p - e)) / (2 * h)
    return g

gc = cxcywh(gt)
print("GT box", gt, " (cx, cy, w, h) =", gc)
cases = {
    "disjoint, 10 px gap":        [gt[2] + 10 + gc[2] / 2, gc[1] + 6, gc[2], gc[3]],
    "disjoint, 60 px gap":        [gt[2] + 60 + gc[2] / 2, gc[1] + 6, gc[2], gc[3]],
    "nested, off-centre":         [gc[0] + 12, gc[1] + 6, gc[2] / 2, gc[3] / 2],
    "nested, concentric, aspect": [gc[0], gc[1], 24.0, 30.0],
}
for name, p in cases.items():
    p = np.array(p, float)
    iou = parts(p)[0]
    v = parts(p)[5]
    alpha0 = v / ((1 - iou) + v)
    losses = {
        "smooth L1": make_smooth_l1(p.copy()),
        "IoU": l_iou, "GIoU": l_giou, "DIoU": l_diou,
        "CIoU (alpha const)": lambda q, a0=alpha0: l_ciou(q, a0),
        "CIoU (alpha diff.)": l_ciou,
    }
    print(f"\n{name}: pred (cx, cy, w, h) = {p}, IoU = {iou:.3f}")
    print("  loss                  value   dL/dcx    dL/dcy    dL/dw     dL/dh")
    for lname, f in losses.items():
        g = grad(f, p)
        print(f"  {lname:19s} {f(p):6.3f}  " + "  ".join(f"{x:+8.4f}" for x in g))

Output:

GT box [30. 42. 90. 78.]  (cx, cy, w, h) = [60. 60. 60. 36.]

disjoint, 10 px gap: pred (cx, cy, w, h) = [130.  66.  60.  36.], IoU = 0.000
  loss                  value   dL/dcx    dL/dcy    dL/dw     dL/dh
  smooth L1            0.681   +0.0167   +0.0046   +0.0000   +0.0000
  IoU                  1.000   +0.0000   +0.0000   +0.0000   +0.0000
  GIoU                 1.209   +0.0061   +0.0188   -0.0036   -0.0016
  DIoU                 1.264   +0.0038   -0.0005   -0.0018   -0.0006
  CIoU (alpha const)   1.264   +0.0038   -0.0005   -0.0018   -0.0006
  CIoU (alpha diff.)   1.264   +0.0038   -0.0005   -0.0018   -0.0006

disjoint, 60 px gap: pred (cx, cy, w, h) = [180.  66.  60.  36.], IoU = 0.000
  loss                  value   dL/dcx    dL/dcy    dL/dw     dL/dh
  smooth L1            1.514   +0.0167   +0.0046   +0.0000   +0.0000
  IoU                  1.000   +0.0000   +0.0000   +0.0000   +0.0000
  GIoU                 1.429   +0.0032   +0.0136   -0.0032   -0.0011
  DIoU                 1.423   +0.0026   -0.0007   -0.0022   -0.0005
  CIoU (alpha const)   1.423   +0.0026   -0.0007   -0.0022   -0.0005
  CIoU (alpha diff.)   1.423   +0.0026   -0.0007   -0.0022   -0.0005

nested, off-centre: pred (cx, cy, w, h) = [72. 66. 30. 18.], IoU = 0.250
  loss                  value   dL/dcx    dL/dcy    dL/dw     dL/dh
  smooth L1            0.616   +0.0133   +0.0185   -0.0231   -0.0385
  IoU                  0.750   +0.0000   +0.0000   -0.0083   -0.0139
  GIoU                 0.750   +0.0000   +0.0000   -0.0083   -0.0139
  DIoU                 0.787   +0.0049   +0.0025   -0.0083   -0.0139
  CIoU (alpha const)   0.787   +0.0049   +0.0025   -0.0083   -0.0139
  CIoU (alpha diff.)   0.787   +0.0049   +0.0025   -0.0083   -0.0139

nested, concentric, aspect: pred (cx, cy, w, h) = [60. 60. 24. 30.], IoU = 0.333
  loss                  value   dL/dcx    dL/dcy    dL/dw     dL/dh
  smooth L1            0.436   +0.0000   +0.0000   -0.0382   -0.0061
  IoU                  0.667   +0.0000   +0.0000   -0.0139   -0.0111
  GIoU                 0.667   +0.0000   +0.0000   -0.0139   -0.0111
  DIoU                 0.667   +0.0000   +0.0000   -0.0139   -0.0111
  CIoU (alpha const)   0.670   +0.0000   +0.0000   -0.0143   -0.0108
  CIoU (alpha diff.)   0.670   +0.0000   +0.0000   -0.0146   -0.0104

Gradient descent moves each parameter against the sign of its derivative, so a positive $\partial L/\partial c_x$ moves a box on the right of the target to the left. In both disjoint cases the IoU loss is exactly flat. Smooth $L_1$ pulls the centre with the same strength at both gaps: the horizontal target error exceeds 1, where its gradient is the constant $1/w_a = 1/60$. The IoU-based penalties weaken as the gap grows, because they are divided by the area or the squared diagonal of the enclosing box. GIoU’s gradient shows the behaviour Zheng et al. described: it grows the box (negative $\partial L/\partial w$), and it pulls vertically more than horizontally. With equal, disjoint boxes the loss is $2 - |B \cup B^{gt}|/|C|$ with a fixed union, and it follows that moving up by one pixel shrinks the $130 \times 42$ enclosing box by its width, 130 square pixels, while moving left shrinks it by only its height, 42. DIoU pulls the centre horizontally but, in this configuration, slightly pushes it further down, and that vertical component is larger at the 60-pixel gap than at the 10-pixel gap. Moving down lengthens the enclosing box’s diagonal $c$ faster than it lengthens the centre distance $\rho$, so $\rho^2/c^2$ falls; the normalization that makes DIoU scale-invariant also lets it decrease without the centres approaching. Differentiating $\rho^2/c^2$ by hand for these two equal boxes (our derivation) shows that the penalty pushes the box away vertically whenever the horizontal centre offset exceeds about 41 pixels, so at every horizontally disjoint position, and that the push is strongest near a 50-pixel gap and fades for larger gaps as $c$ keeps growing. The plan for this example expected that “From near to far gap, GIoU and DIoU gradients shrink”; DIoU’s vertical and width components did not.

In the nested off-centre case, IoU and GIoU give identical values and gradients, and neither moves the centre: the inner box’s IoU is its area divided by the ground truth’s, wherever it sits. DIoU and CIoU add a pull toward the centre, which is the case their paper constructed DIoU for; all the IoU-based losses grow the box at the same rate. In the concentric case, only CIoU sees the aspect-ratio mismatch: it widens the box a little more and heightens it a little less than DIoU, toward the ground truth’s $5:3$ shape. Holding $\alpha$ constant, as the official code does, or differentiating it changes the fourth decimal place here but no sign, and the two choices coincide whenever the aspect ratios already match, since $v$ and its gradient are then zero. Dropping the $1/(w^2 + h^2)$ factor, as the paper’s implementation does, would rescale CIoU’s aspect gradient by $w^2 + h^2$, which depends on the units of $w$ and $h$; it is meant for coordinates normalized to $[0, 1]$ and is not applied in pixel units here. Smooth $L_1$ moves the centre in every case where the centres differ and, in the concentric case, mostly the width, whose log-ratio target ($\log(60/24) \approx 0.92$) is five times the height’s.

Common Misconceptions

• “IoU cannot be a loss because it is not differentiable.” For axis-aligned boxes it is differentiable almost everywhere; its practical problem is the zero gradient for disjoint boxes, which GIoU and DIoU address. Rezatofighi et al. present direct backpropagation of IoU as contrary to the prevailing belief at the time.
• “GIoU always helps when boxes overlap.” Once one box contains the other, GIoU equals IoU, as the second example shows. Its benefit is concentrated in low- and non-overlap cases, which the GIoU authors also give as a reason for smaller gains with dense anchors.
• “An IoU-based loss replaces coordinate losses.” Detectors often use both: DETR adds an $L_1$ loss on normalized coordinates to the GIoU loss, and GFL adds DFL to GIoU. The losses also need re-weighting against classification, so comparisons in papers include a retuned weight.
• “The parameterization does not matter once the loss is IoU-based.” The IoU loss is computed on decoded boxes, but the network still outputs offsets, log scales, distances or bin probabilities, and that choice decides what values the last layer must produce, their range across object sizes and whether decoded sizes stay positive.
• “Regression fixes poor proposals.” Regression is trained on references that already overlap an object: Fast R-CNN applies the box loss only to foreground RoIs, those with IoU at least 0.5, and Tychsen-Smith and Petersson note that IoU of 0.5 limits the centre offset to a sixth of the width and the width ratio to between 1/2 and 2. Bounding box regression refines; finding the object is the job of proposals, anchors and label assignment.

Where It Is Used

Every modern detector regresses boxes. Two-stage detectors such as Faster R-CNN and Mask R-CNN regress twice, in the region proposal network and in the RoI head, classically with smooth $L_1$ on R-CNN offsets; one-stage detectors (SSD, YOLO, RetinaNet) regress offsets from anchors or grid cells. Anchor-free detectors such as FCOS regress side distances with IoU or GIoU losses, and GFL turns those distances into distributions. Set-prediction detectors like DETR regress normalized centre-size boxes from queries with $L_1$ plus GIoU, after matching predictions to objects. Keypoint-based detectors such as CornerNet regress only small offsets that undo the quantization of heatmap peaks. Box overlaps also reappear at inference: DIoU and the learned variances of He et al. modify non-maximum suppression, and average precision on COCO at IoU thresholds up to 0.95 is the metric that rewards precise regression.