FCOS

FCOS, short for Fully Convolutional One-Stage object detector, is an anchor-free object detector that treats detection as per-pixel prediction, in the way a fully convolutional network treats semantic segmentation. At every location of every level of a feature pyramid, it predicts class scores, the four distances from the location to the sides of a box, and a “center-ness” score (the paper’s spelling) that says how close the location is to the centre of that box. Zhi Tian, Chunhua Shen, Hao Chen and Tong He of the University of Adelaide introduced it at ICCV 2019, and a journal version, “FCOS: A Simple and Strong Anchor-free Object Detector”, appeared in IEEE Transactions on Pattern Analysis and Machine Intelligence (online in 2020; volume 44, issue 4, 2022). It is a convolutional, one-stage design in the line of RetinaNet, from which it keeps the ResNet backbone, the five-level pyramid, the head layout and the focal loss, and it removes the anchor boxes. The concept of detection without anchors, and how FCOS compares with keypoint- and set-based designs, is in anchor-free object detection; this article covers the model.

Problem Addressed

In 2019 the leading detectors, Faster R-CNN, SSD, YOLOv2 and v3, and RetinaNet, all regressed boxes from preset anchors. Tian et al. listed four drawbacks: accuracy is sensitive to the anchors’ sizes, aspect ratios and number (by up to 4% AP for RetinaNet on COCO); fixed shapes handle objects of unusual shape poorly and must be redesigned for new datasets; a feature pyramid network tiles more than 180,000 anchors on an image with an 800-pixel shorter side, most of them negatives; and labelling anchors requires the IoU of every anchor with every object.

Per-pixel box prediction already existed. DenseBox (Huang et al., 2015), as Tian et al. describe it, predicts a class and a four-dimensional box vector at every location of one feature map, but crops and resizes training images to a fixed scale and so has to detect on an image pyramid. That family had been used mainly for faces and scene text, and was considered unsuitable for generic detection for two reasons: a location inside two overlapping boxes cannot know which one to regress, and recall was thought to be low, the problem that had led YOLOv2 to add anchors to YOLO. FCOS set out to show that both problems largely disappear when objects are spread over the levels of a feature pyramid by size, and that a remaining accuracy gap, caused by poor boxes from locations far from an object’s centre, can be closed by one extra output.

Architecture

FCOS is a backbone, a feature pyramid and one head shared by all pyramid levels. Figure 2 of the paper gives the sizes for an $800 \times 1024$ input: five maps of $100 \times 128$ (stride 8) down to $7 \times 8$ (stride 128).

Backbone and Feature Pyramid

The backbone is an ImageNet-pretrained ResNet-50 in the ablations, and ResNet-101, ResNeXt-101 or HRNet in the main results. $P_3$, $P_4$ and $P_5$ come from the backbone stages $C_3$ to $C_5$ through $1 \times 1$ lateral convolutions and the top-down pathway of the feature pyramid network. $P_6$ and $P_7$ are made by one stride-2 convolution each, applied to $P_5$ and $P_6$. RetinaNet computes $P_6$ from $C_5$ instead; Tian et al. report that $P_5$ was slightly better (36.4 against 36.3 AP in their Table 3, ResNet-50, COCO minival). The strides of $P_3$ to $P_7$ are 8, 16, 32, 64 and 128.

Head

Each level feeds the same head. As in RetinaNet, a classification tower and a regression tower each apply four $3 \times 3$ convolutions with 256 channels. FCOS adds group normalization after each of these convolutions, which the authors say makes training more stable. The towers end in three outputs per location:

• Classification: $C$ scores, one binary (sigmoid) classifier per class, as in RetinaNet; $C = 80$ on COCO.
• Regression: four values $(l, t, r, b)$, made positive by $\exp(s_i x)$, where $s_i$ is a trainable scalar for level $P_i$. Since the heads are shared but each level regresses a different range of distances, the scalar lets each level adjust the base of the exponential; the authors report a slight gain.
• Center-ness: one value, from a single-layer branch. In the ICCV paper it sits on the classification tower; a footnote added after submission reports that it works better on the regression tower, and this became one of the “improvements”.

With one prediction per location instead of nine anchors, FCOS has nine times fewer output variables than anchor-based detectors with nine anchors per location, such as RetinaNet, as the authors note. In the official code (at commit a773f12), the three output layers are $3 \times 3$ convolutions, all convolutions of the head are initialized from a normal distribution with standard deviation 0.01, and the classification bias is set from a prior probability of 0.01, RetinaNet’s initialization.

Training

Per-Pixel Regression Targets

A location $(x, y)$ of a map with stride $s$ corresponds to the image point $(\lfloor s/2 \rfloor + xs,\ \lfloor s/2 \rfloor + ys)$, near the centre of its receptive field. If that point lies inside a ground-truth box $B_i = (x_0^{(i)}, y_0^{(i)}, x_1^{(i)}, y_1^{(i)})$ of class $c^{(i)}$, the location is a positive with class label $c^{(i)}$, and its regression target is the vector of distances to the four sides (Eq. 1 of the paper):

$$l^* = x - x_0^{(i)}, \qquad t^* = y - y_0^{(i)}, \qquad r^* = x_1^{(i)} - x, \qquad b^* = y_1^{(i)} - y .$$

Every other location is a negative, with class 0. The box is decoded by inverting these equations. The parameterization, its scale behaviour and the losses applied to it are in bounding box regression. Tian et al. stress that FCOS uses as many foreground locations as possible to train the regressor, unlike anchor-based detectors that train only on anchors with high IoU, and suggest this may be one reason it outperforms them.

Multi-Level Assignment by Size

An anchor-based FPN detector sends an object to a level through the sizes of that level’s anchors. FCOS instead limits the range of distances each level may regress. All targets are computed on every level; on level $P_i$ a location becomes a negative if

$$\max(l^*, t^*, r^*, b^*) > m_i \quad \text{or} \quad \max(l^*, t^*, r^*, b^*) < m_{i-1},$$

with $m_2, \ldots, m_7 = 0, 64, 128, 256, 512, \infty$. $P_3$ therefore handles largest distances up to 64 pixels, $P_4$ from 64 to 128, and so on to $P_7$ above 512. A location that still falls inside two boxes on its level, an ambiguous sample, takes the box of minimal area. The authors’ argument is that most overlaps occur between objects of considerably different sizes, which now train different levels. On COCO minival, the share of ambiguous positives fell from 23.16% with $P_4$ alone to 7.14% with five levels, and to 3.75% when overlaps of the same class are not counted, and the best possible recall rose to 98.40%; these measurements and their limits in crowded scenes are discussed in anchor-free object detection, and the rule’s place among assignment methods in label assignment.

The criterion depends on the location, not only on the object. Reading Eq. 1 and the ranges together (our reasoning, checked in the example below): at the centre of an object whose longer side is $L$, the largest distance is about $L/2$; near its edges it approaches $L$. Because the boundaries double from level to level, an object with $L$ between 64 and 1,024 pixels has a boundary $m$ with $L/2 < m < L$, so its central locations can fall on one level and its outer ring on the next coarser one, depending on where the grid points land.

Center-ness

With multi-level prediction, FCOS still trailed anchor-based detectors, and the authors traced the gap to many low-quality boxes predicted from locations far from the centre of their object. Their remedy needs no hyperparameter: a branch predicts, for each positive location, the center-ness

$$\text{centerness}^* = \sqrt{\frac{\min(l^*, r^*)}{\max(l^*, r^*)} \times \frac{\min(t^*, b^*)}{\max(t^*, b^*)}} ,$$

which is 1 at the centre of the box and falls to 0 at its edges; the square root slows the decay (Eq. 3). Because it is a product of ratios, it does not depend on the object’s size. The target lies in $[0, 1]$ and is trained with binary cross-entropy. At test time, the predicted center-ness multiplies the classification score, so boxes from off-centre locations rank lower and are more likely to be removed by non-maximum suppression.

The paper also considered the obvious alternative, computing center-ness from the predicted box $(l, t, r, b)$ instead of learning it. That gave 33.5 AP, no better than no center-ness at all (33.5), while the learned branch gave 37.1 (Table 4, ResNet-50, COCO minival). Tian et al. note a second alternative, used by DenseBox and others: train only on a central portion of each box, at the price of a radius hyperparameter. That alternative later came back as centre sampling (see Variants).

Loss

The training loss (Eq. 2) is

$$L = \frac{1}{N_{\text{pos}}} \sum_{x,y} L_{\text{cls}}(\mathbf{p}_{x,y}, c^*_{x,y}) + \frac{\lambda}{N_{\text{pos}}} \sum_{x,y} \mathbf{1}_{\{c^*_{x,y} > 0\}}\, L_{\text{reg}}(\mathbf{t}_{x,y}, \mathbf{t}^*_{x,y}),$$

summed over the locations of all levels, plus the binary cross-entropy of center-ness. $L_{\text{cls}}$ is the focal loss of RetinaNet (the official code uses $\alpha = 0.25$, $\gamma = 2$), $L_{\text{reg}}$ the IoU loss “as in UnitBox”, $N_{\text{pos}}$ the number of positive locations, and $\lambda = 1$. The IoU of two boxes that share the location can be computed directly from the two distance vectors, so no box is decoded during training.

The official code at commit a773f12 adds details that Eq. 2 does not show. Its default IoU loss is $-\log$ of an IoU smoothed by adding 1 to intersection and union, and it offers $1 - \text{IoU}$ and the GIoU loss. The regression loss of each positive is weighted by its center-ness target and the sum is divided by the sum of those targets rather than by $N_{\text{pos}}$, so central locations count more in regression; the classification and center-ness losses are divided by $N_{\text{pos}}$.

Procedure

The ablation models were trained on COCO trainval35k (about 115,000 images) and evaluated on minival (5,000 images), with RetinaNet’s hyperparameters: stochastic gradient descent for 90,000 iterations with 16 images per batch, learning rate 0.01 divided by 10 at 60,000 and 80,000 iterations, weight decay 0.0001 and momentum 0.9, images resized to an 800-pixel shorter side with the longer side at most 1,333. The models for COCO test-dev were trained for 180,000 iterations with the shorter side drawn at random from 640 to 800 pixels. The README of the official code adds that the ResNet and ResNeXt models were trained with frozen batch normalization, that the authors use 8 NVIDIA V100 GPUs, and that “1x” and “2x” mean 90,000 and 180,000 iterations.

Example: Multi-Level Assignment and Center-ness on a Synthetic Image

The NumPy code below builds FCOS’s training targets for one synthetic image and counts where they fall. An $800 \times 1024$ image, the size used in the paper’s Figure 2, holds eight objects of four classes drawn with scikit-image: a large “table” rectangle, an upright elliptical “person” with a small “ball” overlapping it, a square “box” on the table, two overlapping ellipses of the same class and size (“crowd1” and “crowd2”), a thin vertical “pole” 16 pixels wide, and a $12 \times 12$ “tiny” square. Each box is read from its object’s own mask. Locations follow the official code ($\lfloor s/2 \rfloor + xs$ on maps of $\lceil H/s \rceil \times \lceil W/s \rceil$), and three rules are compared: every stride-16 location inside a box ($P_4$ only, the paper’s no-FPN baseline), the ICCV rule (five levels with the size ranges, inclusive at both ends as in the code, smallest area winning), and the improved configuration’s centre sampling (a square of 1.5 strides around the box centre, clipped to the box, as in the code’s get_sample_region). For each rule it prints positives per object and level, ambiguous positives, objects left with no positive, and the mean center-ness target of positives that lie on the object’s mask and of those that lie inside the box but off the object.

import numpy as np
from skimage.draw import ellipse, disk, rectangle

H, W = 800, 1024
# name, class, shape, parameters (rows/cols in pixels)
objects = [
    ("table",  "C", "rect",    (380, 80, 400, 620)),    # top, left, height, width
    ("person", "A", "ellipse", (420, 820, 170, 60)),    # centre row, col, semi-axes
    ("ball",   "B", "disk",    (560, 790, 22)),
    ("box",    "B", "rect",    (450, 200, 100, 100)),
    ("crowd1", "A", "ellipse", (200, 330, 75, 50)),
    ("crowd2", "A", "ellipse", (210, 400, 75, 50)),
    ("pole",   "D", "rect",    (60, 600, 300, 16)),
    ("tiny",   "B", "rect",    (100, 900, 12, 12)),
]
masks, boxes, cls = [], [], []
for name, c, kind, p in objects:
    m = np.zeros((H, W), bool)
    if kind == "rect":
        rr, cc = rectangle((p[0], p[1]), extent=(p[2], p[3]), shape=(H, W))
    elif kind == "ellipse":
        rr, cc = ellipse(p[0], p[1], p[2], p[3], shape=(H, W))
    else:
        rr, cc = disk((p[0], p[1]), p[2], shape=(H, W))
    m[rr, cc] = True
    ys, xs = np.nonzero(m)
    boxes.append([xs.min(), ys.min(), xs.max() + 1, ys.max() + 1])   # x0, y0, x1, y1
    masks.append(m); cls.append(c)
boxes = np.array(boxes, float); cls = np.array(cls)
area = (boxes[:, 2] - boxes[:, 0]) * (boxes[:, 3] - boxes[:, 1])
names = [o[0] for o in objects]

strides = [8, 16, 32, 64, 128]
ranges = [(-1, 64), (64, 128), (128, 256), (256, 512), (512, 1e8)]   # official code

def locations(s):
    ys, xs = np.meshgrid(np.arange(-(-H // s)) * s + s // 2,
                         np.arange(-(-W // s)) * s + s // 2, indexing="ij")
    return xs.ravel().astype(float), ys.ravel().astype(float)

def targets(xs, ys):
    l = xs[:, None] - boxes[:, 0]; t = ys[:, None] - boxes[:, 1]
    r = boxes[:, 2] - xs[:, None]; b = boxes[:, 3] - ys[:, None]
    return np.stack([l, t, r, b], -1)                     # (locations, objects, 4)

def centre_region(xs, ys, s, radius=1.5):                 # get_sample_region in loss.py
    cx = (boxes[:, 0] + boxes[:, 2]) / 2; cy = (boxes[:, 1] + boxes[:, 3]) / 2
    x0 = np.maximum(cx - radius * s, boxes[:, 0]); x1 = np.minimum(cx + radius * s, boxes[:, 2])
    y0 = np.maximum(cy - radius * s, boxes[:, 1]); y1 = np.minimum(cy + radius * s, boxes[:, 3])
    d = np.stack([xs[:, None] - x0, ys[:, None] - y0, x1 - xs[:, None], y1 - ys[:, None]], -1)
    return d.min(-1) > 0

def assign(rule):
    """Return list of (level, xs, ys, eligible[loc, obj], winner[loc] or -1)."""
    out = []
    levels = [1] if rule == "P4 only" else range(5)
    for i in levels:
        s = strides[i]; xs, ys = locations(s); T = targets(xs, ys)
        ok = T.min(-1) > 0                                 # strictly inside the box
        if rule == "centre sampling":
            ok = centre_region(xs, ys, s)
        if rule != "P4 only":
            mx = T.max(-1)
            ok &= (mx >= ranges[i][0]) & (mx <= ranges[i][1])
        a = np.where(ok, area[None], np.inf)
        win = np.where(np.isfinite(a.min(1)), a.argmin(1), -1)   # smallest area wins
        out.append((i, xs, ys, ok, win, T))
    return out

def centerness(t):
    lr, tb = t[:, [0, 2]], t[:, [1, 3]]
    return np.sqrt(lr.min(1) / lr.max(1) * tb.min(1) / tb.max(1))

for rule in ["P4 only", "FCOS", "centre sampling"]:
    res = assign(rule)
    print(f"== {rule}")
    print("object   class  long side  " + "  ".join(f"P{i+3:<3d}" for i in range(5)) + " total")
    counts = np.zeros((len(names), 5), int)
    for i, xs, ys, ok, win, T in res:
        for k in range(len(names)):
            counts[k, i] = (win == k).sum()
    for k, n in enumerate(names):
        L = int(max(boxes[k, 2] - boxes[k, 0], boxes[k, 3] - boxes[k, 1]))
        print(f"{n:8s} {cls[k]:>5s} {L:10d}  " + "  ".join(f"{c:4d}" for c in counts[k]) + f" {counts[k].sum():5d}")
    pos = amb = amb_diff = 0
    on_c, off_c = [], []
    for i, xs, ys, ok, win, T in res:
        p = win >= 0
        pos += p.sum()
        multi = ok.sum(1) >= 2
        amb += (p & multi).sum()
        diff = np.array([len(set(cls[ok[j]])) > 1 for j in range(len(win))])
        amb_diff += (p & diff).sum()
        idx = np.nonzero(p)[0]
        c = centerness(T[idx, win[idx]])
        onm = np.array([masks[win[j]][min(int(ys[j]), H - 1), min(int(xs[j]), W - 1)] for j in idx], dtype=bool)
        on_c += list(c[onm]); off_c += list(c[~onm])
    print(f"positives {pos}, ambiguous {amb} ({amb / pos:.1%}), different classes {amb_diff} ({amb_diff / pos:.1%}), "
          f"objects without a positive: {[n for k, n in enumerate(names) if counts[k].sum() == 0]}")
    print(f"centerness: on the object mask {len(on_c)} positives, mean {np.mean(on_c):.2f}; "
          f"off the mask {len(off_c)} ({len(off_c) / pos:.1%}), mean {np.mean(off_c) if off_c else float('nan'):.2f}")

Output:

== P4 only
object   class  long side  P3    P4    P5    P6    P7   total
table        C        620     0   939     0     0     0   939
person       A        339     0   141     0     0     0   141
ball         B         43     0     6     0     0     0     6
box          B        100     0    36     0     0     0    36
crowd1       A        149     0    54     0     0     0    54
crowd2       A        149     0    38     0     0     0    38
pole         D        300     0     0     0     0     0     0
tiny         B         12     0     1     0     0     0     1
positives 1215, ambiguous 58 (4.8%), different classes 42 (3.5%), objects without a positive: ['pole']
centerness: on the object mask 1177 positives, mean 0.34; off the mask 38 (3.1%), mean 0.10
== FCOS
object   class  long side  P3    P4    P5    P6    P7   total
table        C        620     0     0     0    36     0    36
person       A        339     0     0    18     4     0    22
ball         B         43    30     0     0     0     0    30
box          B        100    12    34     0     0     0    46
crowd1       A        149     0    42     6     0     0    48
crowd2       A        149     0    30     4     0     0    34
pole         D        300     0     0     0     2     0     2
tiny         B         12     1     0     0     0     0     1
positives 219, ambiguous 14 (6.4%), different classes 0 (0.0%), objects without a positive: []
centerness: on the object mask 200 positives, mean 0.43; off the mask 19 (8.7%), mean 0.11
== centre sampling
object   class  long side  P3    P4    P5    P6    P7   total
table        C        620     0     0     0     9     0     9
person       A        339     0     0     9     0     0     9
ball         B         43     9     0     0     0     0     9
box          B        100     9     7     0     0     0    16
crowd1       A        149     0     9     0     0     0     9
crowd2       A        149     0     9     0     0     0     9
pole         D        300     0     0     0     0     0     0
tiny         B         12     1     0     0     0     0     1
positives 62, ambiguous 0 (0.0%), different classes 0 (0.0%), objects without a positive: ['pole']
centerness: on the object mask 62 positives, mean 0.64; off the mask 0 (0.0%), mean nan

The script runs in well under a second with NumPy 2.3.5 and scikit-image 0.25.2 and has no random elements. One change was made after the first run: the first version stopped with an indexing error when a level had no positives, after printing the first two tables, and the array marking on-mask positives was given a Boolean type; no scene, rule, measurement or printed line changed.

Multi-level assignment cuts the positives from 1,215 to 219, mainly because the table’s 939 locations on $P_4$ shrink to 36 on $P_6$. Four of the six objects with a longer side between 64 and 1,024 pixels are split across two adjacent levels, as the reasoning above predicts: the person trains $P_5$ at its centre and $P_6$ toward its ends, the crowd ellipses $P_4$ and $P_5$, and the $100 \times 100$ box $P_3$ in its middle (12 locations) and $P_4$ in its outer ring (34), so the coarser level gets more of its positives. The table, which by the same argument could reach $P_7$, does not, because none of the twelve stride-128 grid points inside it has a largest distance above 512 pixels (the largest is 508); the pole is kept only through two $P_6$ locations near its ends.

Ambiguity behaves differently from what the plan expected. The plan’s hypothesis read “ambiguous share is much higher with P4 only than with five levels”; counted over all classes it went the other way, from 4.8% to 6.4%. A separate count of which objects share the contested locations (kept with the script) shows why: on $P_4$ alone they come from the table and box (36), the crowd pair (16) and the person and ball (6); with five levels the table, box, person and ball each move to a level of their own, and only the two crowd ellipses, of equal size, still share locations, 14 of them. The count falls by three quarters, but the positives fall more. Counting only overlaps of different classes, as Tian et al. do for their 3.75%, the share drops from 3.5% to zero. The paper’s argument that size separates overlapping objects holds here for every pair except the one of equal size; anchor-free object detection measures the same effect over many random scenes.

Center-ness separates the positives that FCOS’s inside-the-box rule takes from the corners of rounded objects: locations inside the box but off the object have a mean center-ness target of about 0.1, against 0.34–0.43 for locations on the object. These are the locations whose features show mostly background, and at test time their center-ness would rank their boxes low. Centre sampling removes them altogether, leaves nine positives per level for most objects, and leaves no ambiguous location; the tiny square keeps its single location. It also loses the pole. Its central locations have a largest distance of about 150 pixels and belong to $P_5$, whose grid points at stride 32 fall on either side of the 16-pixel-wide pole, while its $P_6$ locations near the ends lie outside the 1.5-stride centre region. In our reading, this is the cost of combining a centre region with a size range defined by distances to the full box: the two rules can disagree about which level an object belongs to, and for a thin object both can miss. The label assignment article’s example found the same failure for a thin pole under centre sampling. The paper’s assignment and the improved one are both printed because the sources describe both as FCOS.

Inference

An image is resized as in training (800-pixel shorter side), passed once through the network, and the outputs of all levels are decoded. Following RetinaNet, the paper keeps locations whose classification score exceeds 0.05 and inverts Eq. 1 to get their boxes; the ranking score is the classification score times the predicted center-ness, and the remaining post-processing uses RetinaNet’s settings. The ICCV results with group normalization use an NMS IoU threshold of 0.6 instead of RetinaNet’s 0.5 (Table 3: 36.5 against 36.4 AP for the model without group normalization).

The official code at commit a773f12 applies the 0.05 threshold to the classification score before multiplying by center-ness, keeps at most 1,000 candidates per level, reports the square root of the product as the score (a monotone change that does not alter the ranking), applies class-wise NMS at 0.6 over all levels together, and keeps the 100 highest-scoring detections per image. The README of that code gives 44 ms per image for the improved ResNet-50 model and claims 6.5 hours of training and 44 ms of inference against 8.8 hours and 56 ms for Faster R-CNN on the same hardware and backbone, without naming the GPU used for timing.

Performance

All numbers below are as reported by the authors, with AP averaged over IoU thresholds 0.5 to 0.95 on COCO (average precision). Ablations use ResNet-50-FPN, the 90,000-iteration schedule and minival; test-dev models use the 180,000-iteration schedule with multi-scale training.

Against RetinaNet (Table 3, minival, ResNet-50-FPN). Trained with RetinaNet’s exact settings (no group normalization, $P_6$ from $C_5$, NMS at 0.5), FCOS reached 36.3 AP against RetinaNet’s 35.9, and 53.5 against 52.5 AR$_{100}$. With $P_5$, group normalization and NMS at 0.6 it reached 37.1. The rows below the paper’s “Improvements” line add, cumulatively, center-ness on the regression branch (37.4), centre sampling (38.1), the GIoU loss (38.3) and regression targets normalized by the level’s stride (38.6). Without the center-ness branch the same model reached 33.5 (Table 4), so center-ness accounted for 3.6 points. Zhang et al. (2020) later compared FCOS with a one-anchor RetinaNet and found that most of the remaining difference came from the definition of positives; that study is described in anchor-free object detection. The ATSS paper attributes to the FCOS paper a comparison of 37.1% against 32.5% for a one-anchor RetinaNet; the ICCV version read here compares only against the standard RetinaNet at 35.9 and gives no one-anchor figure.

COCO test-dev (Table 5, single model, single-scale testing).

Backbone	AP	AP$_{50}$	AP$_{75}$	AP$_S$	AP$_M$	AP$_L$
ResNet-101-FPN	41.5	60.7	45.0	24.4	44.8	51.6
HRNet-W32-5l	42.0	60.4	45.3	25.4	45.0	51.0
ResNeXt-32x8d-101-FPN	42.7	62.2	46.1	26.0	45.6	52.6
ResNeXt-64x4d-101-FPN	43.2	62.8	46.6	26.5	46.2	53.3
ResNeXt-64x4d-101-FPN, with improvements	44.7	64.1	48.4	27.6	47.5	55.6

In the same table, RetinaNet with ResNet-101-FPN has 39.1 AP, 2.4 points below FCOS with the same backbone, CornerNet with Hourglass-104 40.5, FSAF with ResNeXt-64x4d-101-FPN 42.9, and Faster R-CNN with FPN on ResNet-101 36.2. Each baseline row cites the detector’s own paper, so its training schedule and augmentation need not match FCOS’s.

As a region proposal network (Table 6, ResNet-50). Replacing the anchors of an FPN region proposal network with FCOS, with group normalization in the heads and the other settings of the FPN region proposal network in Detectron, raised AR$_{100}$ from 44.7 to 52.8 and AR$_{1k}$ from 56.9 to 60.3, with about 66,000 samples per image against about 200,000 anchors; without center-ness the figures were 48.0 and 59.3.

Released models. The official repository lists, for the original models, minival and test-dev AP of 37.1 and 37.4 (ResNet-50, 1x), 41.4 and 41.5 (ResNet-101, 2x), 42.5 and 42.7 (ResNeXt-32x8d-101, 2x) and 43.0 and 43.2 (ResNeXt-64x4d-101, 2x). Its improved ResNet-50 1x model is listed at 38.7 minival AP, where the paper’s Table 3 prints 38.6 for the same configuration. Models with deformable convolutions (DCNv2) reach 42.3 (ResNet-50, 1x) to 46.6 (ResNeXt-64x4d-101, 2x) minival AP, and the README reports 49.0 AP on test-dev for the largest of them with multi-scale testing.

Strengths

• Fewer design choices. No anchor sizes, aspect ratios or IoU thresholds, and no IoU computation during assignment; the remaining hyperparameters are the level ranges and, in the improved version, the centre-sampling radius.
• Accuracy at equal settings. With RetinaNet’s own hyperparameters, FCOS matched or exceeded it in AP and AR on COCO minival, and with its additions it led the one-stage detectors in its Table 5.
• Many positives per object. Every location inside a box can train the regressor, which the authors suggest helps; the price is low-quality boxes from the edges, handled by center-ness.
• Per-pixel form. Detection takes the same form as segmentation and other dense prediction, so the head can serve as a proposal network, as Table 6 shows, and, according to the authors, be extended to instance segmentation and keypoint detection with little change.

Limitations

• Hand-set level ranges. The ranges 64, 128, 256 and 512 replace anchor scales as a size prior and are fixed by hand; the ATSS authors point out that they remain in the centre-sampling version, and the example shows them splitting single objects across two levels.
• Crowds of similar size. Separation by level relies on overlapping objects differing in size. Objects of the same size share a level, and their shared locations go to the smaller box, which can leave the larger object with few positives. Tian et al. argue that same-class overlaps do not hurt, since either object gives a correct class.
• Thin and tiny objects. An object narrower than the stride of the level its size selects may contain no grid point; the paper’s best possible recall of 98.40% means that about 1.6% of COCO ground-truth boxes received no positive, and the example’s pole was lost under centre sampling.
• NMS remains. Many locations predict each object, so NMS is still needed, and center-ness only re-ranks the duplicates; one-to-one assignment, as in DETR, is the route that removes it.
• A separate quality branch. Center-ness is trained only on positives and multiplied with the class score only at test time, so training and inference score boxes differently. In our reading, this is the inconsistency that later detectors addressed by predicting localization quality jointly with the class score.

Historical Importance

FCOS was one of a group of anchor-free detectors from 2019, with CornerNet (2018), the CenterNet of Zhou, Wang and Krähenbühl (“Objects as Points”), the keypoint-triplet CenterNet of Duan et al., and FoveaBox. It differed from the keypoint detectors in keeping the dense, FPN-based layout of RetinaNet, so it could be compared with an anchor-based detector almost one to one, and the authors encouraged the community to rethink whether anchor boxes are necessary. That comparison led directly to the ATSS study (Zhang et al., 2020), which used FCOS and a one-anchor RetinaNet to show that the definition of positive and negative samples, not the anchor itself, made the difference, and which turned FCOS’s centre and scale rules into the starting point of adaptive label assignment. FCOS also became a common baseline: ATSS reports its results on top of it, and the distribution-based box regression of Generalized Focal Loss and the OTA assigner, as summarized in bounding box regression and label assignment, are also evaluated on FCOS.

Variants

Improvements after the ICCV submission

Table 3 of the ICCV paper and the official code record four changes, each described as nearly cost-free:

• Center-ness on the regression branch. The paper’s acknowledgements credit the suggestion to Chaorui Deng and the README to GitHub user bearcatt; +0.3 AP (37.1 to 37.4).
• Centre sampling. Only locations within 1.5 strides of the box centre, clipped to the box, are positives (CENTER_SAMPLING_RADIUS: 1.5); the size test still uses the distances to the full box. +0.7 AP in Table 3 (37.4 to 38.1). The paper and the README credit this trick and the GIoU loss to the FCOS_PLUS repository of GitHub user yqyao, and the code’s docstring says the function is copied from it; the ATSS paper describes centre sampling as the “lite” version of its own method, merged into the FCOS code, and reports it as raising FCOS from 37.8 to 38.6 AP. The two papers add the improvements in different orders (ATSS applies centre sampling last, to a model that already has the other three), and the final rows coincide: ATSS’s “FCOS + Center sampling” row (38.6, 57.4, 41.4, 22.3, 42.5, 49.8) equals FCOS’s final “+ Normalization” row. Label assignment compares centre sampling with ATSS’s nine nearest candidates.
• GIoU loss. The IoU loss is replaced by $1 - \mathrm{GIoU}$ (Rezatofighi et al., 2019), which subtracts from the IoU the share of the smallest enclosing box not covered by the union of the two boxes; +0.2 AP. Its usual motivation, a gradient for boxes that do not overlap, matters little here in our reading, since a predicted and a true box regressed from the same location both contain it.
• Normalized targets. Regression targets are divided by the level’s stride. In the code, the output then passes through a ReLU after the per-level scalar instead of the exponential, and is multiplied by the stride at test time; +0.3 AP.

The README calls the resulting models imprv and says the changes add about 1.5 AP in total.

Journal version

The journal version, “FCOS: A Simple and Strong Anchor-free Object Detector” (2022), is the reference that the authors’ second code base, AdelaiDet, gives alongside the ICCV paper; its abstract, as deposited in the University of Adelaide’s repository, describes the same per-pixel detector and points to AdelaiDet for the code. Its full text was not read for this article (its arXiv and IEEE copies were not consulted, and the repository record has no public file), so the experiments it adds are not described here. The AdelaiDet code (at commit 5e19cb1) shows the design in its later form: centre sampling with radius 1.5, the GIoU loss, center-ness on the regression tower and ReLU outputs with per-level scalars are all defaults. Its ResNet-50 1x model is listed at 38.7 AP on COCO val2017 and 38.8 on test-dev, and 41.0 and 41.4 with multi-scale training for 2x. An option BOX_QUALITY = "iou" trains the quality branch to predict the IoU of the predicted box instead of center-ness; the code comment says this “generally has ~0.4 better AP”, and the README lists 39.4 AP for the ResNet-50 1x model with it. Real-time models with a DLA-34 backbone and test images with a 512-pixel shorter side are listed at 46 frames per second and 40.3 AP on val2017 (52 frames per second and 39.1 AP with towers shared between classification and regression), timed on an NVIDIA 1080 Ti at batch size 1.

Related detectors

ATSS keeps FCOS’s architecture and replaces its assignment by a per-object IoU threshold over candidates nearest the centre; OTA and later assigners also start from an FCOS-style head. Both are covered in label assignment. The other anchor-free designs of the period, keypoint-based (CornerNet, the two CenterNets) and set-based (DETR), are compared in anchor-free object detection.

Implementations

The authors’ first implementation, tianzhi0549/FCOS, is built on maskrcnn-benchmark and provides the original and imprv configurations with trained models; its README states that it is licensed under the 2-clause BSD licence for academic use and that commercial users should contact the authors (as of November 2026). The second, in AdelaiDet (aim-uofa), is built on Detectron2 and carries the later defaults and the real-time models. The FCOS README also notes an implementation in MMDetection.

TorchVision 0.29 (as of November 2026) provides fcos_resnet50_fpn, citing both the ICCV and the journal paper, with COCO weights that its documentation says were “produced by following a similar training recipe” to the paper’s. The documentation lists 39.2 box mAP on COCO val2017, about 32.3 million parameters and 128.21 GFLOPs, and marks the detection module as beta.