Aerial Object Detection

Aerial object detection finds and classifies objects such as vehicles, ships, aircraft, storage tanks and sports fields in images taken from above, by satellites, aircraft and drones. It is object detection in the overhead imaging domain that remote sensing describes, and the literature also calls it object detection in aerial images, object detection in Earth vision, or, when boxes are rotated, oriented or rotated object detection. Three properties set it apart from detection in photographs: the objects are often a few pixels to a few tens of pixels long and packed side by side, they appear at every orientation because the camera looks down, and the images are far larger than a detector’s input. The task’s own tools follow from these: oriented bounding boxes and their parameterizations, IoU and non-maximum suppression computed on rotated polygons, and tiling with merging. Benchmarks such as DOTA, xView, HRSC2016 and DIOR fix how all of this is scored.

Problem Definition

Given an overhead image, predict for each object of a set of classes its location and a confidence score. Xia et al. (2018), introducing the DOTA benchmark, list what distinguishes the problem from detection in natural images. The scale of objects varies hugely, both because sensors differ in resolution and because sizes vary within a class. Many small instances are crowded together, as ships in a harbour or vehicles in a parking lot, and the number per image is very uneven: some 1k × 1k images in DOTA contain 1,900 instances, while some 4k × 4k images contain only a handful. Objects appear at arbitrary orientations, and some, such as bridges, have extreme aspect ratios. In photographs gravity keeps most objects upright; from above there is no preferred direction.

Location can be required in two forms. An axis-aligned bounding box, also called a horizontal bounding box (HBB) in this literature, is enough for counting or coarse localization. An oriented bounding box (OBB) follows the object’s direction and is required when the objects are elongated and close together, when their heading matters, or when their extent is measured. DOTA defines both tasks on the same images: Task 1 scores oriented boxes against the original oriented annotations, and Task 2 scores axis-aligned boxes against ground truth computed as the axis-aligned boxes of those annotations; the website recommends Task 1 as having more practical value (DOTA website, Tasks page).

Input and Output

The input is usually an RGB or panchromatic image, sometimes with more bands, taken close to nadir, with a known ground sample distance (GSD). DOTA’s images come from Google Earth, the GF-2 and JL-1 satellites and CycloMedia aerial images; the RGB images come from Google Earth and CycloMedia, the grayscale ones from the panchromatic bands of GF-2 and JL-1, and a metadata file gives the acquisition date, source and GSD of each image where known (DOTA website, Dataset page). The GSD is useful information that photographs lack: Xia et al. argue that it lets a model reason about real size rather than pixel size, helps fine-grained distinctions such as small boat against warship, and can flag mislabelled outliers whose size is implausible for their class.

The output is a list of detections, each with a class, a score and a box in one of several parameterizations:

• Eight-point form. The four vertices $(x_1, y_1, \dots, x_4, y_4)$ of a quadrilateral. DOTA annotates every object this way, with the vertices in clockwise order, and for classes with a visible front (planes, ships, vehicles, helicopters, harbours, baseball diamonds) the first vertex marks the object’s “head”; for symmetric classes such as tennis courts it is usually the top-left corner (Xia et al., 2018); the bounding box article gives their reason for preferring quadrilaterals to the angle-based form. Each DOTA annotation line also carries a class and a difficult flag.
• Five-parameter form. $(c_x, c_y, w, h, \theta)$: center, side lengths and a rotation angle. Most detectors regress this form, and it needs a convention, discussed in the next section.
• Other forms. Oriented R-CNN (Xie et al., 2021) uses a six-parameter midpoint offset representation: the axis-aligned box around the object plus the offsets $\Delta\alpha$ and $\Delta\beta$ of two vertices from the midpoints of the top and right sides, which describes a parallelogram that is then turned into an oriented rectangle. Gaussian representations, which turn a box into a two-dimensional Gaussian distribution, and point sets are described under Approaches.

Annotations and predictions must be converted between these forms, and image annotation tools usually export polygons. MMRotate’s documentation notes that polygons exported by annotation software need converting to the rotated-box definition before training (MMRotate documentation, as of November 2026).

Variants

• Horizontal and oriented detection. The two tasks above, often reported side by side.
• Fine-grained detection. Classes split into subtypes. HRSC2016’s ships carry a three-level hierarchy of ship, ship category and ship type (Liu et al., 2017, abstract), and xView’s 60 classes include many vehicle and vessel types (Lam et al., 2018; xView1_baseline README).
• Tiny-object detection. DOTA-v1.5 uses DOTA-v1.0’s images but also annotates extremely small instances, under 10 pixels, and adds the class container crane, for 403,318 instances in total (DOTA website, Home page); the ReDet paper gives 402,089 (Han et al., 2021).
• Other sensors and platforms. Ships are also detected in synthetic aperture radar images, which see through cloud and work at night. Detection in low-altitude drone images and video, with benchmarks such as VisDrone and UAVDT, is described in aerial and drone image analysis, and tracking the detected objects in tracking-by-detection.
• Related outputs. Some overhead datasets give only object centres, as COWC does for cars (Xia et al., 2018, Table 1). Buildings and roads are often extracted as polygons or graphs, and instance masks lead to instance segmentation.

Challenges

Small, dense objects. Xia et al. divided DOTA’s instances by the height of their axis-aligned box into small (10–50 pixels), middle (50–300) and large (above 300): 57%, 41% and 2% of the instances, against 43%, 49% and 8% for COCO and 93% small for the DLR 3K Munich Vehicle dataset. DOTA averages 67.1 annotated boxes per image, against 7.19 for COCO’s 2014 trainval set and 2.89 for PASCAL VOC 07++12, and some images hold more than 1,000. xView’s objects range from 3 m, 10 pixels at its 0.3 m GSD, to more than 3 km (Lam et al., 2018). A small object covers only a few cells of a detector’s downsampled feature map; SCRDet’s authors note that the stride of high-level feature maps tends to skip smaller objects (Yang et al., 2019). The feature pyramid network and the tiling described below are the usual responses.

Orientation and aspect ratio. An elongated object at an oblique angle fills only a small part of its axis-aligned box, and the boxes of parked neighbours overlap heavily. RoI Transformer’s authors observed that one horizontal region of interest often contains several densely packed instances, which confuses the classification and regression that follow (Ding et al., 2019). The example below shows how strongly the overlap of neighbours’ axis-aligned boxes depends on the angle.

Occlusion is rare, crowds are not. Xia et al. point out that from above objects rarely occlude one another, so DOTA annotates every instance in a dense area one by one instead of marking groups as COCO’s “crowd” regions. xView also asked annotators to label each object in a cluster individually, but its authors report that in practice tightly packed groups were sometimes labelled as one object (Lam et al., 2018).

Huge images. DOTA-v1.0’s images range from about 800 × 800 to about 4,000 × 4,000 pixels (Xia et al., 2018, §4.1; their abstract gives about 4,000 × 4,000 for every image); the DOTA website gives up to 20,000 × 20,000 pixels for the current collection, which includes DOTA-v2.0. Detectors take inputs of about 1,000 pixels on a side, so images are cut into tiles and the results merged.

Class imbalance and context. A city holds many thousands more cars than hospitals, and xView keeps that imbalance on purpose (Lam et al., 2018). Its baseline detected planes, set on uniform runways, better than small cars, the class with the second largest instance count. Two classes with more than a thousand instances each, pickup truck and hut/tent, scored under 1% AP in every experiment.

Dataset bias. Imagery differs between sensors, resolutions, regions and seasons. Xia et al., citing earlier work, name the resolution and variety of sensors as causes of dataset bias and collected DOTA from several sources for that reason. In their cross-dataset test, a YOLOv2 detector trained on UCAS-AOD and evaluated on planes and small vehicles reached 89.41 average AP on the UCAS-AOD test images and 53.55 on DOTA, while one trained on DOTA reached 76.15 and 60.51.

Oriented Boxes and Their Problems

The bounding box article shows why the five-parameter form is ambiguous: a rectangle is unchanged by a half-turn, and exchanging $w$ and $h$ while adding 90° gives the same rectangle. Every implementation therefore fixes a convention, and the conventions differ.

Angle conventions

Two families are common. In the OpenCV definition, $\theta$ spans a 90° range and is the angle between the $x$-axis and whichever side meets it at an angle in that range, so the side called the width is chosen by the angle, not by length, and sources differ on which side that is: SCRDet calls the other side the width, MMRotate’s documentation the side that makes the angle. SCRDet uses $\theta \in [-\pi/2, 0)$, which its authors describe as consistent with OpenCV (Yang et al., 2019). In the long-edge definition, $\theta$ is the angle between the longer side and the $x$-axis, over a range of 180°, for example $[-90°, 90°)$. Yang and Yan (2020) call these the 90° and 180° angular ranges. MMRotate’s documentation lists three definitions it supports, switchable by configuration (as of November 2026):

• $D_{oc'}$: angle in $(0°, 90°]$, the range returned by OpenCV’s cv2.minAreaRect from version 4.5.1;
• $D_{le135}$: long edge, angle in $[-45°, 135°)$;
• $D_{le90}$: long edge, angle in $[-90°, 90°)$.

The same documentation notes that before OpenCV 4.5.1 minAreaRect returned angles in $[-90°, 0°)$, so the “OpenCV definition” itself changed with the library version, and that the MMCV operators it uses do not share one direction convention: their rotated IoU and rotated NMS default to clockwise rotation, their rotated RoIAlign operators to counterclockwise. With $y$ pointing down, as in images, a positive angle in the usual rotation matrix turns a box clockwise on screen (our note), as in the examples below. Mixing conventions between annotation conversion, training and evaluation silently rotates or swaps boxes, and the errors grow with aspect ratio, as the second example shows.

The boundary problem

A regression loss on the raw parameters treats them as points on a line, but the angle lives on a circle and the box’s symmetries identify different parameter vectors. Yang et al. (2019) describe the effect in SCRDet: for a ground truth near the end of the angle range, the ideal regression, a small rotation, crosses the range boundary, and the loss on the raw parameters becomes very large. The network is pushed to reach the target another way, by rotating the opposite direction while scaling width and height. Yang and Yan (2020) call this boundary discontinuity, trace it to two causes, the periodicity of the angle and the exchangeability of edges, and show that it affects all three regression families: the 90° form suffers from both, the 180° form from periodicity, and eight-point regression from the ordering of corners, since a correct quadrilateral whose vertices are listed starting from a different corner gets a large loss. They locate the root cause in the ideal prediction lying outside the defined range.

The square-like problem is a second defect, of the long-edge definition. Yang et al. (2021, Gaussian Wasserstein distance) observe that the long-edge definition cannot define a square uniquely: for a nearly square box, a ground truth of $45 \times 43$ at $-60°$ and a prediction of $45 \times 44$ at $30°$ overlap closely (IoU 0.935 in the second example below) but differ by 90° in angle, so the loss is high. Storage tanks and roundabouts in DOTA are such objects. Some detectors choose their convention to avoid one problem: SCRDet and R³Det use the OpenCV definition to avoid the square-like problem, while CSL and its successor DCL use the long-edge definition to avoid the exchangeability of edges (Yang et al., 2021, Gaussian Wasserstein distance).

A third issue is the mismatch between the loss and the metric. Detection is scored by IoU, and a smaller smooth-L1 loss on the parameters does not guarantee a higher IoU. For axis-aligned boxes, IoU-based losses such as GIoU close this gap, as the intersection over union article describes; for rotated boxes the IoU is not differentiable as computed, so these losses do not carry over directly (Yang et al., 2021, Gaussian Wasserstein distance).

Rotated IoU and rotated NMS

The IoU of two oriented boxes is the ratio of the areas of their intersection polygon and their union. The DOTA website states that Task 1 computes IoU as the intersection over union of the two polygons and otherwise follows PASCAL VOC. The DOTA development kit computes polygon intersection areas in C++ (polyiou.cpp), and its evaluation script first computes axis-aligned IoUs as a filter, since boxes whose axis-aligned boxes do not overlap cannot overlap either, and computes polygon IoUs only for the ground truths that pass (DOTA_devkit, commit 9938855). A common route, used in the example below, clips one convex polygon against the other’s edges and takes areas with the shoelace formula.

Non-maximum suppression carries over unchanged once IoU is computed on polygons. SCRDet applies rotation NMS based on the skew IoU as post-processing, with a different threshold for each category because of the diversity of shapes (Yang et al., 2019). The thresholds used in aerial work are often low: DOTA’s baselines merged tile results with NMS at 0.3 for horizontal and 0.1 for oriented boxes (Xia et al., 2018), and Oriented R-CNN applies polygon NMS at 0.1 to each class (Xie et al., 2021). Oriented boxes allow such low thresholds because parked neighbours barely overlap; with axis-aligned boxes the same thresholds delete true neighbours, as the first example shows.

Approaches

The field adapted general detectors, from the two-stage and one-stage families, to the overhead domain. The transitions below were each motivated by a specific failure of the previous step, as the cited papers describe them.

Horizontal detectors and the first oriented baseline

DOTA’s baselines (Xia et al., 2018) tested four general detectors on its horizontal task: Faster R-CNN with ResNet-101 reached 60.46 mAP, R-FCN 52.58, YOLOv2 39.20 and SSD 29.86. The authors suspect that SSD’s random-crop augmentation, useful for natural images, hurts with so many tiny instances. For the oriented task they modified Faster R-CNN to regress the four vertices of the ground-truth quadrilateral relative to each horizontal region of interest. This FR-O reached 54.13 mAP against DOTA’s oriented ground truth, while the horizontally trained detectors, scored against the same oriented ground truth, reached 17.84 to 39.95. In the horizontal task, the authors judged the results for small vehicles, large vehicles and ships far from satisfactory and attribute this to their small size and crowding.

Rotated anchors and learned rotated regions

The next step was to make the regions of interest themselves rotated. Rotated region proposal networks, such as RRPN from scene-text detection, place anchors at many angles as well as scales and aspect ratios (Yang & Yan, 2020; Xie et al., 2021, Figure 1). Ding et al. (2019) argue that this multiplies the number of anchors and the computation, and still cannot match every orientation well. Their RoI Transformer instead learns the transformation from each horizontal region of interest to a rotated one, supervised by the oriented annotations, and extracts rotation-invariant features from the rotated region with a rotated position-sensitive RoI Align. On DOTA’s oriented task, with a ResNet-101 feature pyramid backbone and testing at two scales, they reported 69.56 mAP.

Oriented R-CNN (Xie et al., 2021) simplified the proposal stage again. Its oriented region proposal network is an ordinary RPN whose regression branch outputs six numbers instead of four, the midpoint offset representation above, so oriented proposals come directly from horizontal anchors. The authors reported 75.87 mAP on DOTA with a ResNet-50 FPN backbone (76.28 with ResNet-101, 80.87 with multi-scale training and testing) and, on HRSC2016, 90.40 under the VOC 2007 metric and 96.50 under the VOC 2012 metric with ResNet-50 FPN. They measured 15.1 images per second on one RTX 2080 Ti at 1024 × 1024 input, comparable to the one-stage detectors in their comparison.

Angle as classification

Yang and Yan (2020) avoid the boundary problem by not regressing the angle at all. CSL divides the angle range into bins, 1° wide by default, and treats the angle as a class, which bounds every prediction to the valid range. A one-hot label would make a prediction 1° off as wrong as one 90° off, so they smooth the label with a window function $g$ that is periodic, symmetric and maximal at the true angle $\theta$:

$$\mathrm{CSL}(x) = \begin{cases} g(x), & \theta - r < x < \theta + r, \\ 0, & \text{otherwise}, \end{cases}$$

with radius $r$ and pulse, rectangular, triangular or Gaussian windows; the periodicity makes the label continuous across the boundary. The discretization costs at most half a bin, $\omega/2$, with expected error $\omega/4$. Their DCL successor addresses the square-like problem that the long-edge definition brings (Yang et al., 2021, Gaussian Wasserstein distance).

Losses that couple the parameters

SCRDet (Yang et al., 2019) keeps the smooth-L1 regression but rescales it, per box, so that its magnitude is $|-\log(\mathrm{IoU})|$ while the smooth-L1 term supplies only the gradient direction. At the boundary, where the prediction is nearly right, the IoU is close to 1 and the loss close to 0, which removes the jump. GWD’s authors point out that the gradient direction is still that of smooth L1 and that the box definition must still be checked before computing the IoU.

Yang et al. (2021, Gaussian Wasserstein distance) convert each rotated box into a two-dimensional Gaussian distribution $\mathcal{N}(\mathbf{m}, \Sigma)$ with

$$\mathbf{m} = (c_x, c_y)^\top, \qquad \Sigma^{1/2} = R_\theta \begin{pmatrix} w/2 & 0 \\ 0 & h/2 \end{pmatrix} R_\theta^\top,$$

where $R_\theta$ is the rotation matrix, and use the Wasserstein distance between the two Gaussians,

$$d^2 = \lVert \mathbf{m}_1 - \mathbf{m}_2 \rVert_2^2 + \operatorname{Tr}\!\left( \Sigma_1 + \Sigma_2 - 2 \left( \Sigma_1^{1/2} \Sigma_2 \Sigma_1^{1/2} \right)^{1/2} \right),$$

turned into a loss $1 - 1/(\tau + f(d^2))$ with a nonlinear $f$ and $\tau \ge 1$. The representation has the symmetries of a rectangle built in: $\Sigma^{1/2}(w, h, \theta) = \Sigma^{1/2}(h, w, \theta - \pi/2) = \Sigma^{1/2}(w, h, \theta - \pi)$, and nearly square boxes give nearly the same Gaussian at angles 90° apart. So the boundary and square-like cases produce small distances, whatever angle convention is used, and the distance stays informative when two boxes do not overlap, which the authors note is common for small objects. For two axis-aligned boxes it reduces to an $\ell_2$ loss on $(x, y, w/2, h/2)$.

The same group’s KLD loss (Yang et al., 2021, Kullback–Leibler divergence) replaces the Wasserstein distance with the Kullback–Leibler divergence between the Gaussians. They argue that GWD optimizes the centre independently of the other parameters and is not scale invariant, and prove that KLD is scale invariant; analysing its gradients, they show that it weights the angle error more heavily for objects with large aspect ratios. On HRSC2016 with RetinaNet and the same augmentation, they reported AP at IoU 0.75 of 48.42 with smooth L1, 60.31 with GWD and 72.39 with KLD, so the gains appear mainly at strict IoU thresholds.

Rotation-equivariant features

Ordinary convolutions are not equivariant to rotation, so detectors learn each orientation separately from rotated training data. ReDet (Han et al., 2021) builds the backbone from rotation-equivariant networks, whose features change predictably when the input is rotated, and extracts rotation-invariant features for each rotated region with a rotation-invariant RoI Align that adapts to the region’s orientation. The authors reported 80.10 mAP on DOTA-v1.0 and 76.80 on DOTA-v1.5 with multi-scale training and testing, and a 60% reduction in the number of parameters, which they give as 313 Mb against 121 Mb.

Point sets

Oriented RepPoints (Li et al., 2022) represents each object by a set of adaptive points learned from the image and converts the point set into an oriented box with one of three conversion functions, rather than regressing angles. The authors report results on DOTA, HRSC2016, UCAS-AOD and DIOR-R, and also report a mean average orientation error.

Toolboxes

Most methods above are implemented in open toolboxes. MMRotate provides a unified implementation and evaluation framework for rotated detection and supports the three angle definitions listed above (MMRotate documentation, as of November 2026).

Tiling Large Images

A 4,000-pixel image cannot be fed to a detector at full resolution, and shrinking it would erase the small objects. The standard procedure, a form of the tiled inference in sliding window detection, has three parts.

Splitting for training. Xia et al. (2018) cropped 1024 × 1024 patches with a stride of 512. An object cut by a patch border keeps its original label if at least 70% of its area falls inside the patch and is marked difficult otherwise, and its clipped part is refitted to a four-vertex box. The development kit’s ImgSplit.py implements this with a default patch size of 1024, an overlap (gap) of 100 pixels and an area threshold of 0.7; clipped parts with five vertices are reduced to four by merging the shortest edge, and parts with more vertices are dropped (DOTA_devkit, commit 9938855). Later papers chose their own settings: RoI Transformer and ReDet crop 1024 × 1024 training patches with a stride of 824, RoI Transformer at two image scales (testing with a stride of 512) and ReDet, for its multi-scale results, at three (Ding et al., 2019; Han et al., 2021). The xView baseline cut its images, about 3,000 × 3,000 pixels per square kilometre, into non-overlapping 300 × 300 chips, cropping boxes at chip edges (Lam et al., 2018).

Detecting per tile. Each patch is processed on its own, sometimes at several scales. Lam et al. trained SSD on chips of 300, 400 and 500 pixels and evaluated on the 300-pixel test chips; the multi-resolution training set gave 0.2590 mAP against 0.1456 for single-size chips and 0.1549 with added augmentation.

Merging. Detections are shifted back by their patch’s offset and divided by its scale; in the devkit’s ResultMerge.py both are parsed from the patch file names. Objects in the overlap are detected twice, and per-class NMS on polygons removes the duplicates, with a default threshold of 0.3 in the devkit and 0.1 for the oriented baselines in Xia et al. (2018). DOTA’s evaluation, the devkit notes, is on the large images, so a submission is always merged first. Overlap must exceed the length of the largest object to guarantee that each object lies whole in some patch (our reasoning); Xia et al. report bridges as large as 1,200 pixels, far more than a 100-pixel overlap.

Example: Oriented and Axis-Aligned Boxes in a Synthetic Parking Scene

The example builds a 512 × 512 synthetic overhead image with three lots of long vehicles, 40 × 6 pixels parked side by side with 2-pixel gaps and pointing at 0°, 30° and 45°, plus twelve 20 × 8 pixel cars at random angles. A minimal “detector” thresholds the image, labels connected components and fits an oriented box to each from its second moments: a uniform rectangle of side $L$ has variance $L^2/12$ along that side. Each detection gets one duplicate shifted 2 pixels along its axis with 0.9 times its score, a simulated second response of the kind NMS must remove. Rotated IoU is computed by Sutherland–Hodgman clipping. It then compares the oriented and axis-aligned representations: how well each fits, how much neighbours overlap, how NMS behaves, and how matches are counted at IoU above 0.5, the DOTA devkit’s rule. It needs NumPy 2.3.5, SciPy 1.16.3 and scikit-image 0.25.2.

import numpy as np
from scipy import ndimage
from skimage.draw import polygon as fill_polygon

def corners(cx, cy, l, w, t):
    """Corners of an oriented box: center, length l along angle t (radians), width w."""
    u = np.array([np.cos(t), np.sin(t)]) * l / 2       # half-length vector
    v = np.array([-np.sin(t), np.cos(t)]) * w / 2      # half-width vector
    c = np.array([cx, cy])
    return np.array([c - u - v, c + u - v, c + u + v, c - u + v])

def cross(a, b):                                       # z-component of the 2D cross product
    return a[..., 0] * b[..., 1] - a[..., 1] * b[..., 0]

def area(p):                                           # shoelace formula
    x, y = p[:, 0], p[:, 1]
    return 0.5 * abs(np.dot(x, np.roll(y, -1)) - np.dot(y, np.roll(x, -1)))

def clip(subject, clipper):
    """Sutherland-Hodgman: part of convex polygon `subject` inside convex `clipper`."""
    out = subject
    s = np.sign(cross(clipper[1] - clipper[0], clipper[2] - clipper[1]))  # orientation
    for a, b in zip(clipper, np.roll(clipper, -1, axis=0)):
        inp, out = out, []
        if len(inp) == 0:
            break
        inside = lambda p: s * cross(b - a, p - a) >= -1e-9   # tolerance for collinear edges
        for p, q in zip(inp, np.roll(inp, -1, axis=0)):
            if inside(q):
                if not inside(p):
                    out.append(intersect(p, q, a, b))
                out.append(q)
            elif inside(p):
                out.append(intersect(p, q, a, b))
        out = np.array(out)
    return out

def intersect(p, q, a, b):
    d1, d2 = q - p, b - a
    t = cross(a - p, d2) / cross(d1, d2)
    return p + t * d1

def iou_poly(p, q):
    inter = clip(p, q)
    i = area(inter) if len(inter) >= 3 else 0.0
    return i / (area(p) + area(q) - i)

def hbb(p):                                            # axis-aligned box of a polygon, as a polygon
    x0, y0 = p.min(0); x1, y1 = p.max(0)
    return np.array([[x0, y0], [x1, y0], [x1, y1], [x0, y1]])

# Sanity checks of the rotated IoU.
sq = corners(0, 0, 1, 1, 0)
print("IoU checks:", iou_poly(sq, sq), round(iou_poly(sq, corners(0, 0, 1, 1, np.pi / 4)), 4),
      "expected 1 and", round((2 * np.sqrt(2) - 2) / (2 - (2 * np.sqrt(2) - 2)), 4))

# Synthetic overhead scene: three lots of long vehicles (40 x 6 px, gap 2 px) and scattered cars.
rng = np.random.default_rng(0)
H = W = 512
gt, group = [], []
for g, (deg, x0, y0) in enumerate([(0, 30, 40), (30, 300, 40), (45, 120, 260)]):
    t = np.deg2rad(deg)
    along = np.array([-np.sin(t), np.cos(t)])          # row direction, across the vehicles
    axis = np.array([np.cos(t), np.sin(t)])             # vehicle long axis
    for r in range(2):
        for k in range(10):
            c = np.array([x0, y0]) + k * 8 * along + r * 46 * axis
            gt.append(corners(*c, 40, 6, t)); group.append(f"{deg} deg lot")
while sum(gr == "cars" for gr in group) < 12:          # scattered cars, no overlaps
    c = rng.uniform([280, 300], [490, 490]); t = rng.uniform(0, np.pi)
    p = corners(*c, 20, 8, t)
    if all(iou_poly(p, q) == 0 for q in gt):
        gt.append(p); group.append("cars")
group = np.array(group)

img = np.full((H, W), 0.2)
for p in gt:
    rr, cc = fill_polygon(p[:, 1], p[:, 0], (H, W))
    img[rr, cc] = 0.8
img += rng.normal(0, 0.05, img.shape)

# "Detector": threshold, 4-connected components, oriented box from second moments.
lab, n = ndimage.label(img > 0.5)
print("objects:", len(gt), "components:", n)
dets, scores, src = [], [], []
for i in range(1, n + 1):
    yy, xx = np.nonzero(lab == i)
    cx, cy = xx.mean(), yy.mean()                       # skimage.draw: pixel centers at integers
    lam, vec = np.linalg.eigh(np.cov(np.vstack([xx, yy])))
    t = np.arctan2(vec[1, 1], vec[0, 1])
    l, w = np.sqrt(12 * lam[1]), np.sqrt(12 * lam[0])  # uniform rectangle: var = side^2 / 12
    dets.append(corners(cx, cy, l, w, t)); scores.append(img[yy, xx].mean())
    src.append(max(range(len(gt)), key=lambda j: iou_poly(dets[-1], gt[j])))
    shift = 2 * np.array([np.cos(t), np.sin(t)])        # simulated duplicate response
    dets.append(dets[-1] + shift); scores.append(0.9 * scores[-1]); src.append(src[-1])
scores, src = np.array(scores), np.array(src)

def nms(boxes, scores, thr, iou):
    order, keep = list(np.argsort(-scores)), []
    while order:
        i = order.pop(0); keep.append(i)
        order = [j for j in order if iou(boxes[i], boxes[j]) <= thr]
    return keep

def match(dets, scores, gts, iou, thr=0.5):
    """VOC-style greedy matching in score order; a match needs IoU > thr (DOTA devkit)."""
    taken, tp = np.zeros(len(gts), bool), []
    for i in np.argsort(-scores):
        ious = [iou(dets[i], g) for g in gts]
        j = int(np.argmax(ious))
        ok = ious[j] > thr and not taken[j]
        taken[j] |= ok; tp.append((i, ok))
    return tp, taken

iou_hbb = lambda p, q: iou_poly(hbb(p), hbb(q))
groups = ["0 deg lot", "30 deg lot", "45 deg lot", "cars"]
print("\ngroup        IoU(OBB,own HBB)  max HBB IoU of neighbours  IoU(estimated OBB, GT)")
for gname in groups:
    idx = np.nonzero(group == gname)[0]
    fit = np.mean([iou_poly(gt[j], hbb(gt[j])) for j in idx])
    nb = max(iou_hbb(gt[a], gt[b]) for a in idx for b in idx if a < b)
    est = np.mean([iou_poly(dets[2 * k], gt[src[2 * k]]) for k in range(n) if src[2 * k] in idx])
    print(f"{gname:12s} {fit:16.3f} {nb:26.3f} {est:23.3f}")

print("\nNMS on", len(dets), "candidates (", n, "objects + one duplicate each )")
print("threshold  kept(R-NMS)  kept(HBB-NMS)")
for thr in [0.3, 0.5, 0.7]:
    print(f"{thr:9.1f} {len(nms(dets, scores, thr, iou_poly)):12d} {len(nms(dets, scores, thr, iou_hbb)):14d}")

print("\nMatching at IoU > 0.5 after NMS at 0.5: TP / FP / FN per group")
for name, iou in [("OBB task, R-NMS  ", iou_poly), ("HBB task, HBB-NMS", iou_hbb)]:
    keep = nms(dets, scores, 0.5, iou)
    tp, taken = match([dets[k] for k in keep], scores[keep], gt, iou)
    row = []
    for gname in groups:
        in_g = group == gname
        t = sum(ok for i, ok in tp if in_g[src[keep[i]]])
        f = sum(not ok for i, ok in tp if in_g[src[keep[i]]])
        row.append(f"{gname}: {t}/{f}/{int((in_g & ~taken).sum())}")
    print(name, " | ".join(row))

Output:

IoU checks: 1.0 0.7071 expected 1 and 0.7071
objects: 72 components: 72

group        IoU(OBB,own HBB)  max HBB IoU of neighbours  IoU(estimated OBB, GT)
0 deg lot               1.000                      0.000                   0.842
30 deg lot              0.253                      0.479                   0.989
45 deg lot              0.227                      0.518                   0.934
cars                    0.657                      0.000                   0.966

NMS on 144 candidates ( 72 objects + one duplicate each )
threshold  kept(R-NMS)  kept(HBB-NMS)
      0.3           72             52
      0.5           72             62
      0.7           72             72

Matching at IoU > 0.5 after NMS at 0.5: TP / FP / FN per group
OBB task, R-NMS   0 deg lot: 20/0/0 | 30 deg lot: 20/0/0 | 45 deg lot: 20/0/0 | cars: 12/0/0
HBB task, HBB-NMS 0 deg lot: 20/0/0 | 30 deg lot: 20/0/0 | 45 deg lot: 10/0/10 | cars: 12/0/0

The IoU of an oriented box with its own axis-aligned box is the share of the axis-aligned box that the object fills, $wh / \big((w|\cos\theta| + h|\sin\theta|)(w|\sin\theta| + h|\cos\theta|)\big)$, which follows from the enclosing-box widths in the bounding box article. For the 40 × 6 vehicles it falls to about a quarter at 30° and 45°: three quarters of each axis-aligned box is ground or neighbouring vehicles. Neighbours’ oriented boxes do not overlap at all, but their axis-aligned boxes overlap with IoU 0.479 at 30° and 0.518 at 45°, on either side of the usual threshold of 0.5.

That margin decides NMS. Rotated NMS keeps 72 boxes at every threshold, and at 0.5 the matching confirms one per object: a duplicate overlaps its original strongly (for an exact 40 × 6 box, a 2-pixel shift along its length gives IoU $38/42 \approx 0.90$, our calculation), while neighbours’ oriented boxes do not overlap at all. NMS on axis-aligned boxes needs a threshold between the neighbours’ overlap and the duplicates’. At 0.3, DOTA’s horizontal merge threshold, it keeps only 52 boxes for 72 objects; at 0.5 it keeps 62, and the matching shows that the ten lost vehicles are all in the 45° lot; at 0.7 it keeps 72 here, but by the same geometry it would fail for tighter packing or longer objects (our reasoning). The matching step itself is not the problem: every detection that survives NMS is a true positive in both tasks, and the ten missing vehicles count as false negatives because they were suppressed before scoring.

The last column checks the moment-based estimator. At 30° and 45°, and for the cars, the estimated boxes match the ground truth with IoU 0.93–0.99. In the 0° lot it is only 0.842. Our explanation, consistent with that value, is that scikit-image’s polygon rasterizer also fills the pixels whose centres lie exactly on the box’s edges, so each 40 × 6 vehicle drawn with edges on integer coordinates covers 41 × 7 pixels; the moments of a uniform 41 × 7 block give a box of about 41.0 × 6.9, whose IoU with the true box is about 0.845. One pixel on a 6-pixel-wide object costs about 16% of the IoU, and the same happens whenever annotation and prediction disagree on pixel conventions, the bounding box article’s point, magnified for small objects (our reasoning).

The script was changed after its first run. The first run added half a pixel to every estimated centre, assuming pixel centres at half-integers, while scikit-image’s rasterizer puts them at integers; removing the offset changed only the last column of the first table. The first run also printed NumPy deprecation warnings for two-dimensional cross products and a divide-by-zero warning for collinear edges in the clipper, so np.cross was replaced by the explicit cross function and the inside test given a tolerance of $10^{-9}$; no printed value changed. The lots use elongated vehicles, decided before the first run, because a hand calculation showed that the axis-aligned boxes of side-by-side 20 × 8 cars with 2-pixel gaps overlap with IoU below about 0.3 at any angle.

Example: Angle Error and the Boundary Problem

The second example uses corners and iou_poly from the first. It measures how an angle error lowers IoU for boxes of different aspect ratios, and how the long-edge parameterization treats nearly identical boxes near the boundary and square-like boxes, compared with the rotated IoU and the Gaussian Wasserstein distance of Yang et al. (2021).

import numpy as np
from scipy.linalg import sqrtm
# Uses corners() and iou_poly() from the first example.

# 1. IoU of a box and a copy rotated about its center, by aspect ratio.
deltas = [0.25, 0.5, 1, 2, 5, 10, 15]
print("angle error (deg):", "  ".join(f"{d:5}" for d in deltas))
for ar in [1, 4, 9]:
    box = corners(0, 0, 10 * ar, 10, 0)
    ious = [iou_poly(box, corners(0, 0, 10 * ar, 10, np.deg2rad(d))) for d in deltas]
    print(f"aspect 1:{ar}        ", "  ".join(f"{v:5.3f}" for v in ious))

# 2. The boundary problem under the long-edge definition: theta in [-90, 90) deg, w = long side.
def le90(cx, cy, w, h, deg):
    if h > w:                                           # long side becomes w
        w, h, deg = h, w, deg + 90
    deg = (deg + 90) % 180 - 90                          # wrap into [-90, 90)
    return np.array([cx, cy, w, h, deg])

def gwd(a, b):
    """Gaussian Wasserstein distance (Yang et al., 2021): box -> N(m, Sigma), Sigma^1/2 = R diag(w/2, h/2) R^T."""
    def gauss(cx, cy, w, h, deg):
        t = np.deg2rad(deg)
        R = np.array([[np.cos(t), -np.sin(t)], [np.sin(t), np.cos(t)]])
        return np.array([cx, cy]), R @ np.diag([w ** 2 / 4, h ** 2 / 4]) @ R.T
    m1, S1 = gauss(*a); m2, S2 = gauss(*b)
    r1 = sqrtm(S1).real
    d2 = np.sum((m1 - m2) ** 2) + np.trace(S1 + S2 - 2 * sqrtm(r1 @ S2 @ r1).real)
    return np.sqrt(max(d2, 0.0))

cases = {"long box rotated 89 -> 91 deg": ((0, 0, 40, 6, 89), (0, 0, 40, 6, 91)),
         "same box, 1 deg rotation    ": ((0, 0, 40, 6, 10), (0, 0, 40, 6, 11)),
         "square-like 45x43 vs 45x44  ": ((0, 0, 45, 43, -60), (0, 0, 45, 44, 30))}
print("\ncase                           le90 parameters (a | b)                    L1(params)  IoU    GWD")
for name, (a, b) in cases.items():
    pa, pb = le90(*a), le90(*b)
    iou = iou_poly(corners(*pa[:2], *pa[2:4], np.deg2rad(pa[4])), corners(*pb[:2], *pb[2:4], np.deg2rad(pb[4])))
    print(f"{name} {np.array2string(pa, precision=0)} | {np.array2string(pb, precision=0)}"
          f"  {np.abs(pa - pb).sum():8.1f}  {iou:5.3f}  {gwd(pa, pb):5.2f}")

Output:

angle error (deg):  0.25    0.5      1      2      5     10     15
aspect 1:1         0.996  0.991  0.983  0.967  0.923  0.863  0.816
aspect 1:4         0.991  0.982  0.964  0.930  0.836  0.705  0.594
aspect 1:9         0.980  0.961  0.924  0.854  0.672  0.440  0.273

case                           le90 parameters (a | b)                    L1(params)  IoU    GWD
long box rotated 89 -> 91 deg [ 0  0 40  6 89] | [  0   0  40   6 -89]     178.0  0.889   0.67
same box, 1 deg rotation     [ 0  0 40  6 10] | [ 0  0 40  6 11]       1.0  0.943   0.34
square-like 45x43 vs 45x44   [  0   0  45  43 -60] | [ 0  0 45 44 30]      91.0  0.935   1.12

Aspect ratio amplifies angle error. A square loses only 14% of its IoU at 10°, while a 1:9 box, the shape of a ship or a bridge, falls to 0.440 and would be scored as a false positive and a missed object at the 0.5 threshold. For the 1:9 box the table gives IoU drops of 0.020 at 0.25° and 0.039 at 0.5°; Yang and Yan (2020), discussing the discretization error of CSL for the same aspect ratio, give 0.02 and 0.05. KLD’s authors give this sensitivity of objects with large aspect ratios as the motivation for their loss, and their HRSC2016 results, quoted above, separate the losses mainly at strict IoU thresholds.

In the boundary case, rotating a 40 × 6 box from 89° to 91° wraps its long-edge angle to −89°, and the raw parameters differ by 178 (pixels and degrees summed, as an unweighted regression loss would), although the two boxes are 2° apart and overlap with IoU 0.889. A 1° rotation away from the boundary, from 10° to 11°, changes the parameters by 1. The square-like pair overlaps with IoU 0.935 yet differs by 91, nearly all of it the 90° angle difference. The Gaussian Wasserstein distance stays small in all three cases, 0.34 to 1.12 pixels, because the Gaussian has the rectangle’s symmetries; in the square-like case it measures only the difference in side lengths, 45 × 43 against 44 × 45 once the 90° relabelling is undone. No change was made to this script after its first run, which already used the corrected functions of the first example.

Datasets and Benchmarks

The table summarizes the main benchmarks. Facts come from each dataset’s paper or official pages; where those could not be read, the source is named.

Dataset	Imagery	Annotation	Size
NWPU VHR-10	Images about 1000 px wide	Horizontal boxes, 10 classes	800 images, 3,651 instances (as tabulated by Xia et al., 2018)
VEDAI	Images 512 and 1024 px wide	Oriented boxes, 3 main classes	1,268 images, 2,950 instances (Xia et al., 2018)
UCAS-AOD	Images about 1000 px wide	Oriented boxes, planes and vehicles	1,510 images, 14,596 instances (Xia et al., 2018)
HRSC2016	Harbour scenes, 300 × 300 to 1500 × 900 px	Oriented boxes, three-level ship classes	1,061 images; 436 training, 181 validation, 444 test (Xie et al., 2021; Yang & Yan, 2020)
DOTA-v1.0	Google Earth, GF-2, JL-1, CycloMedia	Quadrilaterals, 15 classes	2,806 images, 188,282 instances
xView	WorldView-3, 0.3 m GSD	Horizontal boxes, 60 classes	About 1 million objects over more than 1,400 km²
DOTA-v1.5	As DOTA-v1.0	Adds instances under 10 px and container crane, 16 classes	2,806 images, 403,318 instances
DIOR-R	DIOR images, 800 × 800 px	Oriented boxes, 20 classes	23,463 images, 192,518 instances (as described by Li et al., 2022)
DOTA-v2.0	More Google Earth, GF-2 and aerial images	Adds airport and helipad, 18 classes	11,268 images, 1,793,658 instances

DOTA. DOTA-v1.0 splits its images at random into one half for training, one sixth for validation and one third for testing, with test annotations withheld and results submitted to an evaluation server (Xia et al., 2018). Experts in aerial image interpretation chose the classes and annotated the images, and their annotations include stuff-like harbours alongside objects. DOTA-v2.0 adds images and splits them into 1,830 training images (268,627 instances), 593 validation images (81,048), 2,792 test-dev images (353,346) and 6,053 test-challenge images (1,090,637); the website says the training and validation shares were kept smaller than the test sets to avoid overfitting, and that DOTA-v1.0’s images were relabelled for v2.0, so v2.0 models must use only v2.0 annotations (DOTA website, as of November 2026). Images and annotations may be used for academic purposes only; the remote sensing article gives the licence and other properties.

xView. Annotators drew axis-aligned boxes in QGIS on 1 km² chips, falling back to a parent class when unsure of a fine-grained one, and skipped objects less than about 20% visible. Quality control had three stages, the last of which required batches to reach precision 0.75 and recall 0.95 at IoU 0.5 against expert gold-standard labels, otherwise they were re-annotated (Lam et al., 2018). The public release has train, validation and test splits made per image, holding 59.2%, 20.8% and 20.0% of the objects; the authors note that splitting by image made class proportions hard to balance, because objects are spatially correlated.

HRSC2016. The dataset paper describes ship images with horizontal and rotated boxes, a three-level class hierarchy, the head position of ships with V-shaped bows and segmentation masks for the test set (Liu et al., 2017, abstract). The full paper could not be read; the image counts and splits above come from the papers that use the dataset, and Yang and Yan (2020) describe its images as coming from six harbours, with ships at sea and inshore.

DIOR. DIOR-R adds oriented box annotations to the images of the DIOR dataset. Neither the DIOR nor the DIOR-R paper could be read here; the counts in the table are as Li et al. (2022) report them for DIOR-R.

Results on these benchmarks are comparable only under the same protocol. The cited papers differ in backbone, image scales, tiling stride, rotation augmentation, whether training and validation sets were combined for training, and the AP variant. Oriented R-CNN’s HRSC2016 results show the size of the last factor: 90.40 under the VOC 2007 metric and 96.50 under the VOC 2012 metric for the same model (Xie et al., 2021).

Evaluation Metrics

Aerial benchmarks use average precision per class and its mean, with matches decided by intersection over union on the box type of the task. The details, which the AP and IoU articles define in general, differ between benchmarks:

• DOTA. Task 1 matches on polygon IoU, Task 2 on axis-aligned IoU; otherwise the protocol follows PASCAL VOC (DOTA website, Tasks page). In the devkit’s evaluation script a detection is matched when its IoU with the best-overlapping ground truth is strictly greater than 0.5, a ground truth already matched turns further detections into false positives, objects marked difficult count neither way, and the script’s main function calls the 11-point VOC 2007 AP (DOTA_devkit, commit 9938855). The website asks that no class file in a submission be empty.
• HRSC2016. Papers report AP under the VOC 2007 11-point metric, the VOC 2012 all-point metric, or both (Xie et al., 2021; Yang & Yan, 2020).
• xView. The baseline scoring code matches at IoU at least 0.5 and computes AP by the PASCAL VOC method. Each detection is compared only with the ground truth it overlaps most; if that one is already taken, the detection is a false positive even if it overlaps another ground truth above the threshold. Besides overall mAP, the code reports mAP for small, medium and large classes and for common and rare classes, where the groups are fixed lists of classes, unlike COCO’s size ranges of individual instances (xView1_baseline, commit 8efe340). In our reading of the code, the ground truths of a class in an image with no detections of that class are not added to that class’s count of ground truths, so such misses do not lower its recall.
• Orientation. Some papers add an orientation error, such as Oriented RepPoints’ mean average orientation error (mAOE) (Li et al., 2022), or AP at strict thresholds, such as AP at IoU 0.75 or averaged over 0.5–0.95 in the KLD paper (Yang et al., 2021), because AP at 0.5 tolerates sizeable angle errors on compact objects (our reasoning, consistent with the second example).

Precision and recall at a fixed score threshold are used when counts matter, as in vehicle counting.

Applications

Xia et al. (2018) name vehicle counting and remote object tracking among the uses that motivate DOTA, and Yang and Yan (2020) object-level change detection, for which orientation and scale information help. xView’s authors mention applied research in census mapping (Lam et al., 2018), and the xView website states that interviews with disaster-response and public-safety experts informed its 60-class ontology. Ship detection in harbours is the setting of HRSC2016. Detections feed counts and inventories of parked vehicles, aircraft or storage tanks, maritime monitoring, and the land-use mapping of land cover classification. Very high-resolution imagery can reveal people’s property and activities, and the remote sensing article discusses the privacy and dual-use questions that come with it.

Open Problems

• Tiny objects. DOTA-v1.5’s added container crane class scored 11.53 AP single-scale and 46.98 multi-scale for ReDet, far below the other classes (Han et al., 2021). Objects of a few pixels carry little appearance information, and a one-pixel error already costs much of their IoU, as the 0° lot of the first example shows (our reasoning).
• Protocols. Angle conventions, AP variants, tiling and multi-scale testing differ between papers, which makes reported numbers hard to compare. Test sets with withheld labels depend on evaluation servers: the DOTA website notes that its server is still improving, offers evaluation by email as an alternative, and says that this manual evaluation takes about two days (as of November 2026).
• Box shape. Rectangles enclose irregular objects such as harbours and sector-shaped baseball diamonds only loosely, which is why DOTA annotated quadrilaterals. Extreme aspect ratios remain the hardest case for angle estimation, as the second example shows.
• Generalization. Detectors trained on one sensor, resolution or region degrade on another, as the cross-dataset test of Xia et al. (2018) showed, and some fine-grained xView classes were hard for its baseline even with more than a thousand examples.
• Scale of the image. Tiling trades context for resolution: objects cut by tile borders, objects larger than the overlap, and the cost of processing images of tens of thousands of pixels on a side are handled by heuristics rather than learned.