Cell Detection

Cell detection finds every cell, nucleus or other cellular object of interest in a microscopy or histology image and reports each as a point, usually its centre, for questions that need only a location per object: how many cells, where, and which belong to a rare class such as dividing cells. Cell segmentation gives each cell a mask; detection gives up the mask and in exchange needs cheaper annotation (one click per cell instead of an outline), simpler outputs and a different way of scoring, in which a prediction counts as correct if it lies close enough to a reference point. Mitosis detection in tumour histology, where the objects are mitotic figures among thousands of ordinary nuclei, is the variant with the longest series of public challenges. Compared with general object detection, which predicts boxes and classes, cell detection deals with many small, similar, often touching objects whose extent matters less than their position.

Problem Definition

Given an image, predict a set of points $D = \{d_1, \dots, d_M\}$, optionally with a class and a confidence for each, such that each object of interest is represented by exactly one point and no point lies on something else. Cireşan et al. (2013) state the mitosis version this way: find the set of detections, each reporting the centroid of a single mitosis. The reference is a set of annotated points $G = \{g_1, \dots, g_N\}$, and $N$ is not known in advance.

Because a centre is not a well-defined pixel (an annotator clicks somewhere inside the nucleus, and a dividing cell may appear as two dark pieces), the task definition always includes a tolerance: a prediction matches a reference point if it lies within a radius $r$ of it. The radius is part of the problem, not only of the evaluation. AMIDA13 set it to 7.5 μm (30 pixels at 0.25 μm per pixel) and used the same radius both to merge the two pathologists’ annotations into one reference and to score the algorithms (Veta et al., 2014, arXiv version).

Counting is the coarser problem of estimating $N$ alone, for the whole image or for any region of it. It can be solved by detecting and counting, or without localizing any object (Lempitsky & Zisserman, 2010).

Input and Output

The input is a 2D image, sometimes a 3D stack or a time-lapse sequence:

• Fluorescence images of stained nuclei or of labelled subcellular structures, recorded by fluorescence microscopy, in which objects appear as bright blobs on a dark background, the case that blob detectors assume.
• Brightfield images of stained tissue, most often hematoxylin and eosin (H&E), in which nuclei are dark purple. Mitosis datasets are regions of whole-slide images: AMIDA13 distributed 2,000 × 2,000-pixel tiles, each about one high-power field (Veta et al., 2014, arXiv version), and MIDOG used regions of 2 mm² scanned by several slide scanners (Aubreville et al., 2022, arXiv version).

The output is a list of coordinates, each with a score and, where classes exist, a label. A threshold on the score turns a ranked list into a detection set; benchmarks that report precision–recall curves keep the scores. Annotations are point lists as well: the AMIDA13 training labels were the centres of the pathologists’ ellipses (Veta et al., 2014, arXiv version), and the MIDOG organizers supplied the labels as square boxes of equal size, 50 pixels (Aubreville et al., 2022, arXiv version).

Variants

• Nucleus and cell detection. Every nucleus is a target, often as the first step towards counting or classifying cells. Sirinukunwattana et al. (2016) detect all nuclei in colon cancer histology and then classify each as epithelial, inflammatory, fibroblast or miscellaneous.
• Spot detection. Small bright objects in fluorescence images, such as labelled particles or subcellular structures. Matula et al. (2015) note that particles are treated as volumeless, so the overlap test used to match cells has no equivalent for them.
• Mitosis detection. Only nuclei in division are targets. Mitotic figures are rare, and other dark structures (apoptotic or compressed nuclei, debris) look similar (Veta et al., 2014, arXiv version).
• Counting without detection. The output is a count or a density map, as in Lempitsky and Zisserman (2010).
• Detection for tracking. In time-lapse microscopy, detections are linked over time; the graph-based accuracy measure used in the Cell Tracking Challenge has a detection-only variant, AOGM-D (Matula et al., 2015).

Challenges

• Crowding and scale. Two touching nuclei form one bright region, so methods that threshold and take one point per region merge them, and a filter searching scales up to the size of a pair responds to it as to one object. Nucleus size in pixels also depends on magnification and varies within an image. The example below measures both effects.
• Ambiguous references. In AMIDA13 the two pathologists marked 1,088 and 1,599 objects and agreed on 649, a Dice overlap of 0.483 between their annotation sets before tiles were selected; a panel of two further pathologists decided the rest (Veta et al., 2014, arXiv version). When a second two-observer panel later re-examined the false positives of the best algorithm, it judged 61 of its 208 false positives to be mitoses.
• Rarity and look-alikes. Cireşan et al. (2013) labelled about 66,000 pixels near mitosis centroids against 151 million others in their training images, and most of the negatives were trivial background, so they resampled the training set to over-represent hard negatives.
• Domain shift. Stain, scanner and tumour type change the appearance of nuclei. The MIDOG organizers built two challenges around this shift (Aubreville et al., 2022, arXiv version; Aubreville et al., 2024, arXiv version).

Approaches

Blob detectors: LoG and DoG

The classical detector for round, bright objects is the scale-normalized Laplacian of Gaussian. Filtering the image $I$ with $-\sigma^2 \nabla^2 G_\sigma$ gives a strong positive response at the centre of a bright blob whose size matches $\sigma$; the factor $\sigma^2$ makes responses at different scales comparable, and the Laplacian operator article derives why the response peaks at a scale proportional to the blob’s radius. A detector computes this response for a list of scales, stacks the results into a scale-space volume, and keeps points that are local maxima in both position and scale and exceed a threshold. A difference of Gaussians (DoG), the image blurred at scale $\sigma$ minus the image blurred at $k\sigma$, approximates filtering with $-(k - 1)\sigma^2 \nabla^2 G_\sigma$, so it gives the same detector more cheaply when a stack of Gaussian-blurred images is computed anyway; SIFT uses it for keypoints.

scikit-image’s implementations (release 0.25.2) show the choices that matter in practice. blob_log evaluates num_sigma scales between min_sigma and max_sigma, multiplies each negated Laplacian-of-Gaussian image by $\sigma^2$, and finds local maxima over a $3 \times 3 \times 3$ neighbourhood of position and scale that exceed threshold, an absolute value on the normalized response. blob_dog builds a geometric series of scales with ratio sigma_ratio (default 1.6) and divides each difference by $k - 1$ so its magnitude matches the normalized Laplacian. Both then prune: when two detected blobs overlap by more than the fraction overlap of the smaller one’s area, the smaller is removed. The documentation gives each blob’s radius as about $\sqrt{2}\,\sigma$ in 2D. Both functions assume bright objects on a dark background, so stained nuclei in brightfield images must be inverted or unmixed into a nuclear channel first. The scale range is a prior on object size, the threshold a prior on contrast, and the overlap rule a prior on how close objects may be.

Detection from segmentation

Any instance segmentation gives detections: the centroid of each object. Thresholding followed by connected components, or a distance-transform watershed that splits touching nuclei, are described in the cell segmentation article. Sirinukunwattana et al. (2016) used one such pipeline, CRImage (thresholding, morphological operations, distance transform and watershed, then centroids), as a baseline. Its detections inherit every merge and split of the segmentation.

Pixel classifiers and peak picking

A learned alternative classifies every pixel as near a centre or not and then reduces the resulting probability map to points. Cireşan et al. (2013), who won the ICPR 2012 mitosis contest, trained deep max-pooling convolutional networks on $101 \times 101$-pixel RGB windows to say whether the central pixel lay within $d = 10$ pixels of a mitosis centroid. On a test image they averaged the probability maps of two networks applied to eight rotated and mirrored copies, smoothed the result with a disk of radius $d$, and then repeatedly took the highest remaining pixel as a detection and zeroed everything within $2d$ of it, a non-maximum suppression on points, until no pixel exceeded a threshold chosen on validation images. Their AMIDA13 entry used the same scheme with a 20-pixel target disk and suppression in a 50-pixel radius (Veta et al., 2014, arXiv version).

Most other AMIDA13 entries worked in two stages: threshold a nuclear or “blue ratio” image to find dark candidate objects, then classify each candidate from colour, texture and shape features or a patch-based network, with random forests and support vector machines the commonest classifiers (Veta et al., 2014, arXiv version). The two-stage design survived into the deep-learning era. In MIDOG 2021, several teams detected candidates with Mask R-CNN, Faster R-CNN or DetectorRS and classified them with a second network, while others used one-stage detectors or segmentation networks (Aubreville et al., 2022, arXiv version, Table 1).

Proximity maps

Kainz et al. (2015) observed that a pixel classifier trained to mark centres produces several local maxima around one nucleus and that smoothing them away can merge neighbours. They regressed instead a proximity score that falls with the distance $D_C(x)$ from pixel $x$ to the nearest annotated centre:

$$d(x) = \begin{cases} e^{\alpha \left(1 - D_C(x)/d_M\right)} - 1 & \text{if } D_C(x) < d_M, \\ 0 & \text{otherwise}, \end{cases}$$

which is flat on the background and peaks sharply at centres; $d_M$ is chosen so the peak width matches the object size ($\alpha = 5$, $d_M = 16$ on one dataset and $\alpha = 3$, $d_M = 39$ on the other). A regression random forest predicted $d(x)$ from 21 feature channels, and detections were the local maxima above a threshold. The spatially constrained CNN of Sirinukunwattana et al. (2016) regresses a similar target, $1/(1 + \|z - z_m\|^2/2)$ within a radius $d = 4$ pixels of the nearest centre $z_m$, but its final layers estimate the centre coordinates and peak heights of up to $M$ such masks in each patch and draw them, so the output map has the required shape by construction.

Heatmap regression is also how general detection and pose estimation predict points. CenterNet (Zhou et al., 2019) splats each object centre onto a heatmap with a Gaussian whose width depends on object size, trains with a focal loss, and keeps the local peaks of the predicted heatmap, which a $3 \times 3$ max-pooling finds without box-based suppression; pose estimation predicts body joints the same way.

Density maps and counting

Lempitsky and Zisserman (2010) separated counting from detection. From dot annotations they defined a ground-truth density as a sum of normalized Gaussians centred on the dots,

$$F^0(p) = \sum_{P \in \mathcal{P}} \mathcal{N}\!\left(p;\, P,\, \sigma^2 \mathbf{1}_{2 \times 2}\right),$$

so that integrating the density over any region gives the number of objects in it, and learned a linear map from per-pixel features to density. Their loss, the MESA distance, is the largest absolute difference between the two densities’ sums over all axis-aligned boxes in the image; it ignores local noise and jitter that cancel over larger regions but reacts to missing or displaced objects, and it can be computed with a maximum-subarray algorithm. They argued that counting by detection fails where instances overlap, and that counting by regression from global image features discards the positions in the annotation and so needs many more training images.

Their synthetic fluorescence images of bacterial cells, generated with a cell-image simulator (the test images averaged 171 ± 64 cells), became a standard counting benchmark (Cohen et al., 2017, arXiv version). Later networks replaced the linear map with fully convolutional ones; Count-ception (Cohen et al., 2017, arXiv version) predicts, for every $32 \times 32$ window, the number of cells inside it, so each cell is counted many times, and averages the redundant counts. A density map gives counts without positions; its local maxima can be read as detections, but nothing in the training asks them to coincide with objects one to one.

Domain generalization in mitosis detection

The organizers’ MIDOG 2021 reference method was a RetinaNet, a one-stage detector, trained on the 50-pixel boxes, with a domain classifier attached through a gradient reversal layer so that the learned features carried less information about the scanner; the same network without that branch, trained with standard augmentation only, served as a baseline (Aubreville et al., 2022, arXiv version). Participants used stain augmentation or stain normalization, synthesis of other scanners’ appearance with generative networks, Fourier-domain mixing, ensembles and test-time augmentation; the organizers observed that five of the seven best approaches used ensembling or test-time augmentation. For MIDOG 2022, which added tumour types, species and laboratories, the organizers report that the top approaches of 2021 had all used segmentation (Aubreville et al., 2024, arXiv version).

Example: spot detection scored by point matching

The script (NumPy 2.3.5, SciPy 1.16.3, scikit-image 0.25.2) draws five $256 \times 256$ fluorescence-like images, each with 80 elliptical nuclei whose long semi-axis is 4.5 to 6.5 pixels and short semi-axis 0.75 to 1 times as long. Each new nucleus has a 40% chance of being placed in contact with an earlier one (centre distance 0.85 to 1.0 times the sum of their mean radii); the others keep a gap. Brightness varies per nucleus, illumination falls from 1.0 on the left to 0.7 on the right, and the image is blurred ($\sigma = 1.5$ pixels) with Gaussian noise ($\sigma = 0.03$). The true centres are known. Three detectors run on each image: blob_log with scales 2 to 6, blob_dog with the same range, and an Otsu threshold followed by connected components, one centroid per component. Detections are matched one to one to true centres within $r = 5$ pixels, about one nucleus radius, by a linear assignment that maximizes the number of pairs (SciPy’s linear_sum_assignment, which solves the assignment problem of the Hungarian algorithm). The script reports precision, recall and F1 with counts pooled over the five images, the count error, the median distance of matched pairs, and an F1 under a lenient rule that follows the wording of AMIDA13, in which a true centre counts as found if any detection lies within $r$.

The parameters were fixed in a plan before the first run. Two things were added after it, following a diagnosis of the first output, and the first run’s numbers are unchanged by them: the last printed line, recall for nuclei placed in contact with another against isolated ones, and the fourth detector row, blob_log with the scale range narrowed to 2 to 4, added to test the explanation below; a diagnostic run had tried this range, the only alternative tried, before it was added. The radius sweep uses the original blob_log row.

import sys

import numpy as np
from scipy import ndimage as ndi
from scipy.optimize import linear_sum_assignment
from skimage.feature import blob_dog, blob_log
from skimage.filters import threshold_otsu

H = W = 256

def synth(seed, n=80):
    """Elliptical nuclei (many touching) under uneven illumination; returns image, centres, touching flags."""
    rng = np.random.default_rng(seed)
    yy, xx = np.mgrid[:H, :W] + 0.5
    nuc = []                                                # (row, col, a, b, angle)
    while len(nuc) < n:
        a = rng.uniform(4.5, 6.5)
        b = a * rng.uniform(0.75, 1.0)
        r = (a + b) / 2
        if nuc and rng.random() < 0.4:                      # touch an earlier nucleus
            q = nuc[rng.integers(len(nuc))]
            phi = rng.uniform(0, 2 * np.pi)
            d = rng.uniform(0.85, 1.0) * ((q[2] + q[3]) / 2 + r)
            p = np.array(q[:2]) + d * np.array([np.sin(phi), np.cos(phi)])
            lo = 0.85
        else:
            p = rng.uniform(8, H - 8, 2)
            lo = 1.1
        if not (8 <= p[0] <= H - 8 and 8 <= p[1] <= W - 8):
            continue
        if all(np.hypot(*(p - o[:2])) >= lo * ((o[2] + o[3]) / 2 + r) - 1e-9
               for o in nuc if o is not (q if lo == 0.85 else None)):
            nuc.append((p[0], p[1], a, b, rng.uniform(0, np.pi)))
    signal = np.zeros((H, W))
    for cy, cx, a, b, th in nuc:
        u = (xx - cx) * np.cos(th) + (yy - cy) * np.sin(th)
        v = -(xx - cx) * np.sin(th) + (yy - cy) * np.cos(th)
        inside = (u / a) ** 2 + (v / b) ** 2 <= 1
        signal[inside] = np.maximum(signal[inside], 0.8 * rng.lognormal(0, 0.3))
    light = 1.0 - 0.3 * xx / W                              # illumination 1.0 -> 0.7
    img = ndi.gaussian_filter(light * signal + 0.1, 1.5)   # optical blur
    img += rng.normal(0, 0.03, img.shape)
    c = np.array([o[:2] for o in nuc])
    rad = np.array([(o[2] + o[3]) / 2 for o in nuc])
    dist = np.hypot(*(c[:, None] - c[None]).transpose(2, 0, 1)) + np.eye(n) * 1e9
    touching = (dist <= 1.05 * (rad[:, None] + rad[None])).any(1)   # placed in contact
    return img, c, touching

def detect(img):
    log = blob_log(img, min_sigma=2, max_sigma=6, num_sigma=9, threshold=0.1, overlap=0.5)
    log4 = blob_log(img, min_sigma=2, max_sigma=4, num_sigma=9, threshold=0.1, overlap=0.5)
    dog = blob_dog(img, min_sigma=2, max_sigma=6, sigma_ratio=1.6, threshold=0.1, overlap=0.5)
    sm = ndi.gaussian_filter(img, 1.0)
    lab, _ = ndi.label(ndi.binary_fill_holes(sm > threshold_otsu(sm)))
    sizes = np.bincount(lab.ravel())
    keep = [i for i in range(1, lab.max() + 1) if sizes[i] >= 10]
    cents = np.array(ndi.center_of_mass(np.ones_like(img), lab, keep)) + 0.5
    return {"LoG (blob_log)": log[:, :2] + 0.5, "DoG (blob_dog)": dog[:, :2] + 0.5,
            "Otsu + components": cents, "LoG, max_sigma 4": log4[:, :2] + 0.5}

def match(gt, det, r):
    """One-to-one matching within radius r maximizing the number of pairs."""
    if len(gt) == 0 or len(det) == 0:
        return 0, len(det), len(gt), np.array([]), np.zeros(len(gt), bool)
    d = np.hypot(*(gt[:, None, :] - det[None, :, :]).transpose(2, 0, 1))
    cost = np.where(d <= r, d / (r * 1e3), 1.0)            # any pair within r beats none
    i, j = linear_sum_assignment(cost)
    ok = d[i, j] <= r
    tp = int(ok.sum())
    hit = np.zeros(len(gt), bool)
    hit[i[ok]] = True
    return tp, len(det) - tp, len(gt) - tp, d[i, j][ok], hit

def lenient(gt, det, r):
    """AMIDA13-style wording: a true point is found if any detection lies within r."""
    d = np.hypot(*(gt[:, None, :] - det[None, :, :]).transpose(2, 0, 1))
    found = int((d <= r).any(1).sum())
    fp = int((~(d <= r).any(0)).sum())
    return found, fp, len(det)

def f1(tp, fp, fn):
    return 2 * tp / (2 * tp + fp + fn)

first = int(sys.argv[1]) if len(sys.argv) > 1 else 0
seeds = range(first, first + 5)
R = 5.0
radii = [1, 2, 3, 5, 8]
tot, abs_err, dists, len_tot, sweep = {}, {}, {}, {}, np.zeros((len(radii), 3), int)
n_true, hits, touch = 0, {}, []
for seed in seeds:
    img, gt, touching = synth(seed)
    n_true += len(gt)
    touch.extend(touching)
    for name, det in detect(img).items():
        tp, fp, fn, dd, hit = match(gt, det, R)
        hits.setdefault(name, []).extend(hit)
        tot[name] = tot.get(name, np.zeros(3, int)) + (tp, fp, fn)
        abs_err.setdefault(name, []).append(abs(len(det) - len(gt)))
        dists.setdefault(name, []).extend(dd)
        found, lfp, nd = lenient(gt, det, R)
        len_tot[name] = len_tot.get(name, np.zeros(3, int)) + (found, lfp, nd)
        if name == "LoG (blob_log)":
            for k, r in enumerate(radii):
                sweep[k] += match(gt, det, r)[:3]

print(f"images {seeds.start}-{seeds.stop - 1}: {n_true} true nuclei; match radius {R:g} px")
print(f"{'method':18s} {'found':>5s} {'TP':>4s} {'FP':>4s} {'FN':>4s} {'prec':>6s} {'recall':>6s}"
      f" {'F1':>6s} {'count err':>9s} {'mean|err|':>9s} {'med dist':>8s} {'F1 lenient':>10s}")
for name, (tp, fp, fn) in tot.items():
    found, lfp, nd = len_tot[name]
    lp, lr = (nd - lfp) / nd, found / n_true
    print(f"{name:18s} {tp + fp:5d} {tp:4d} {fp:4d} {fn:4d} {tp / (tp + fp):6.3f} {tp / (tp + fn):6.3f}"
          f" {f1(tp, fp, fn):6.3f} {tp + fp - n_true:+9d} {np.mean(abs_err[name]):9.1f}"
          f" {np.median(dists[name]):8.2f} {2 * lp * lr / (lp + lr):10.3f}")
print("LoG, one-to-one F1 by match radius:",
      ", ".join(f"r={r:g}: {f1(*s):.3f}" for r, s in zip(radii, sweep)))
touch = np.array(touch)
print(f"recall by neighbourhood ({touch.sum()} touching, {(~touch).sum()} isolated nuclei):",
      "; ".join(f"{name} {np.mean(np.array(h)[touch]):.3f} / {np.mean(np.array(h)[~touch]):.3f}"
                for name, h in hits.items()))

Output:

images 0-4: 400 true nuclei; match radius 5 px
method             found   TP   FP   FN   prec recall     F1 count err mean|err| med dist F1 lenient
LoG (blob_log)       328  328    0   72  1.000  0.820  0.901       -72      14.4     0.49      0.901
DoG (blob_dog)       321  321    0   79  1.000  0.802  0.890       -79      15.8     0.48      0.892
Otsu + components    236  222   14  178  0.941  0.555  0.698      -164      32.8     0.18      0.714
LoG, max_sigma 4     357  357    0   43  1.000  0.892  0.943       -43       8.6     0.49      0.943
LoG, one-to-one F1 by match radius: r=1: 0.777, r=2: 0.860, r=3: 0.871, r=5: 0.901, r=8: 0.901
recall by neighbourhood (221 touching, 179 isolated nuclei): LoG (blob_log) 0.674 / 1.000; DoG (blob_dog) 0.643 / 1.000; Otsu + components 0.285 / 0.888; LoG, max_sigma 4 0.805 / 1.000

Every error of the blob detectors is a miss, and every miss is a nucleus touching another: both find all 179 isolated nuclei and about two thirds of the touching ones. Our explanation is the scale range. Two touching nuclei form a bright region about twice as wide as one, and when the range includes scales near the pair’s size, the normalized Laplacian can respond more strongly to the pair than to either nucleus, so the scale-space maximum sits on the pair rather than on each nucleus. The blob detectors made no false positives, so each detection on a merged pair lay within 5 pixels of one of its nuclei, and a merge appears as a miss rather than as a false positive. Narrowing the range to $\sigma \le 4$, which still includes the best scale of most of these nuclei (about $\rho/\sqrt{2}$ for a disk of radius $\rho$, a little less after the blur; mean radii here run from 3.9 to 6.5 pixels), lifts recall on touching nuclei from 0.67 to 0.81 with no false positives, consistent with this explanation. The scale range is a size prior, and setting it from the expected nucleus size rather than generously costs nothing here; a population with very varied sizes would not allow that.

Thresholding with connected components is far worse on touching nuclei (recall 0.29) because a merged region yields one centroid, and it also misses about one in nine isolated nuclei, presumably faint ones on the dim side of the image where a single global threshold falls above them. Its centroids are the most precise when they are right (median error 0.18 pixels against about 0.5 for the blob detectors), presumably because a centroid averages many pixels while blob peaks lie on integer pixel positions. Its lenient F1 is higher than its one-to-one F1 because one centroid between two touching nuclei is within 5 pixels of both and is credited with finding both; the blob detectors produce almost no such in-between points (one, for blob_dog), so the two rules nearly agree for them. The radius sweep shows the opposite sensitivity: F1 rises from 0.78 at $r = 1$ to 0.90 at $r = 5$ and then stops changing, so on these images a radius below about 3 pixels scores localization noise rather than detection.

The count error equals $\mathrm{FP} - \mathrm{FN}$, so it says nothing about errors that cancel; here false positives are almost absent and the count error is just the number of misses. Running the script with first seeds 5 and 10 (images 5–9 and 10–14) gave F1 of 0.877 and 0.906 for blob_log, 0.866 and 0.907 for blob_dog, 0.650 and 0.648 for thresholding, and 0.932 and 0.946 for the narrowed blob_log, with all isolated nuclei found by the blob detectors and recall on touching nuclei of 0.64 to 0.71 for the wide scale range and 0.80 to 0.83 for the narrow one.

Datasets and Benchmarks

MITOS (ICPR 2012). The first publicly available mitosis dataset (Aubreville et al., 2024, arXiv version), used for the contest at the International Conference on Pattern Recognition in 2012: 50 high-power fields from five H&E breast cancer slides, about 300 mitoses, each field 512 × 512 μm² and acquired with two slide scanners and a multispectral microscope; the Aperio images are 2,084 × 2,084 pixels at 0.2456 μm per pixel (Cireşan et al., 2013). Fifteen fields formed the test set, and a detection counted if it lay within 5 μm (20 pixels) of a reference centroid. The AMIDA13 organizers noted that regions of the same slides appeared in training and test data (Veta et al., 2014, arXiv version).

AMIDA13. Held with MICCAI 2013: 23 consecutive invasive breast carcinoma cases from the University Medical Center Utrecht, one slide each, scanned at 0.25 μm per pixel, split by patient into 12 training and 11 test subjects; 1,157 consensus mitoses from two independent pathologists and a two-pathologist panel for discordant objects; test labels withheld and three submissions per team (Veta et al., 2014, arXiv version). Of the 14 teams that submitted, 11 methods are described in the paper.

TUPAC16. A challenge on predicting proliferation scores from whole-slide images, with an auxiliary mitosis detection dataset of 73 breast cancer cases from three centres (23 of them from AMIDA13) and 1,552 annotated mitotic figures (Veta et al., 2019, arXiv version).

MIDOG 2021. A MICCAI 2021 challenge on scanner shift: 200 training cases of human breast cancer, 50 each from four scanners (one of them without labels, for unsupervised domain adaptation), and a final test set of 80 cases from four scanners, two of them not in the training set (Aubreville et al., 2022, arXiv version). The abstract gives 100 test cases, which matches the 20-case preliminary set plus the 80-case final set described in the text. Labels came from one pathologist’s screening, a detector run at low threshold to find missed candidates, a blinded second pathologist and a third for disagreements. Participants submitted Docker containers evaluated on hidden data, with one final submission.

MIDOG 2022. The 2022 edition added tumour types, species (human, canine, feline) and laboratories: 405 training cases with 9,501 mitotic-figure annotations in six domains, five of them labelled, and a final test set of 100 cases in ten tumour domains disjoint from those of the preliminary test set (Aubreville et al., 2024, arXiv version). The test reference was a three-expert majority vote on H&E, with a second reference assisted by immunohistochemistry for phospho-histone H3 (PHH3). An extended training set was released under a CC0 licence after the challenge; the challenge page (as of October 2026) announces it as MIDOG++.

Nucleus detection sets. CRCHisto (Sirinukunwattana et al., 2016) has 100 H&E images of 500 × 500 pixels at 0.55 μm per pixel from colorectal adenocarcinoma, with 29,756 nuclei marked at their centres and 22,444 of them classified. Kainz et al. (2015) released a bone marrow set of eleven 1,200 × 1,200 H&E images with 4,205 dot annotations, and used 20 small breast cancer images from the ICPR 2010 histopathology contest. The synthetic bacterial-cell images of Lempitsky and Zisserman (2010) serve as a counting benchmark.

Cell Tracking Challenge. Scores detection in time-lapse data by matching through masks rather than distances; see Evaluation Metrics and the cell segmentation article.

Reported results

Each row uses its own data, matching radius and pooling, so rows compare only within a source.

Source	Data and protocol	Reported result
Cireşan et al. (2013)	MITOS, 15 test fields, 5 μm radius	F1: DNN 0.782 (precision 0.88, recall 0.70); next best contestant 0.718
Veta et al. (2014, arXiv)	AMIDA13, 11 test subjects, 7.5 μm, counts pooled	F1: IDSIA 0.611, DTU 0.483, third 0.344; per-subject average F1: 0.445, 0.352
Veta et al. (2019, arXiv)	TUPAC16 mitosis task (matching protocol not given in this paper)	top F1 0.652, against 0.612 for the AMIDA13 winner (the AMIDA13 paper prints 0.611)
Aubreville et al. (2022, arXiv)	MIDOG 2021, 80 test cases, four scanners, 7.5 μm, pooled	F1: winner 0.748 (95% CI 0.704–0.781), organizers’ reference 0.718, baseline without domain branch 0.698
Aubreville et al. (2024, arXiv)	MIDOG 2022, 100 test cases, ten tumour domains, track 1	F1: winner 0.764, runner-up 0.757
Sirinukunwattana et al. (2016)	CRCHisto, two-fold cross-validation, 6-pixel radius	F1: SC-CNN 0.802 ($M = 2$), centre-of-the-pixel CNN 0.692, CRImage 0.542
Kainz et al. (2015)	bone marrow set, leave-one-out cross-validation	F1: regression forest 87.17 ± 2.73, Arteta et al.’s method 84.30 ± 3.28
Lempitsky & Zisserman (2010)	synthetic cells, 100 test images, 32 training images	mean absolute count error: density learning 3.5 ± 0.2, a detection-based baseline 8.5 ± 0.5
Cohen et al. (2017, arXiv)	same synthetic cells, 32 training images	mean absolute count error: Count-ception 2.4 ± 0.4, Lempitsky and Zisserman 3.5 ± 0.2

In MIDOG 2021 the best results on scanner D (in the training set without labels) and on the unseen scanner F were considerably weaker than on the others (the winner’s F1 was 0.728 and 0.634 there, against 0.793 and 0.781 on scanners A and E), which the organizers relate to domain shift not fully covered by the algorithms or to the scanners’ image quality. In MIDOG 2022, domain characteristics absent from training (a new species, spindle-cell morphology, a new scanner) led to small but significant decreases (Aubreville et al., 2024, arXiv version).

Evaluation Metrics

Point detection is scored by matching predictions to reference points within a radius and counting true positives (matched predictions), false positives (unmatched predictions) and false negatives (unmatched references), from which precision, recall and the F1 score follow. Veta et al. (2014, arXiv version) note that F1 here equals the Dice overlap between the set of detections and the set of reference objects, which lets an algorithm’s score be compared directly with the agreement between two annotators. Several details change the number.

The radius. It is usually set to about one object size: 7.5 μm in AMIDA13 and MIDOG 2021, the average size of a mitotic figure in those datasets; 5 μm in MITOS; 6 pixels at 0.55 μm per pixel in CRCHisto. Kainz et al. (2015) showed why it must be reported: on their bone marrow data, tightening it from 16 to 8 pixels lowered the area under the precision–recall curve of a competing method from 89.9 to 86.9 while theirs went from 90.7 to 90.5, and they inferred that an earlier paper, which stated no radius, must have used a larger one than their 4 pixels on the ICPR 2010 images.

Duplicates and matching. Rules for several detections near one reference differ. AMIDA13 counted several detections within 7.5 μm of one reference as one true positive (Veta et al., 2014, arXiv version), while MIDOG 2021 counted the extra ones as false positives, since they would inflate the mitotic count (Aubreville et al., 2022, arXiv version). Kainz et al. (2015) assigned the most confident detection to the reference, Sirinukunwattana et al. (2016) the closest. The MIDOG 2021 evaluation code scores each case with score_detection from evalutils 0.2.4 with radius=7.5E-3, the paper’s 7.5 μm if, as that value implies, the coordinates are in millimetres; that function takes the reference points in order and gives each the nearest prediction within the radius not already used. This greedy rule can produce fewer pairs than the maximum-cardinality assignment used in the example when neighbouring references compete for the same predictions (our reasoning); with objects much farther apart than the radius, as mitoses usually are, the two agree. Its docstring says that several predictions hitting one reference count as one true positive, but the code counts every unused prediction as a false positive, as the MIDOG paper describes.

Pooling. AMIDA13 reported both an overall F1, with all reference objects treated as one set, and the average of per-subject F1 scores; the winner scored 0.611 and 0.445, because subjects with few mitoses weigh heavily in the average, and one test subject had no mitoses, so its F1 was undefined (Veta et al., 2014, arXiv version). MIDOG computes F1 from true positives, false positives and false negatives summed over all cases, so that low-prevalence slides are not over-represented, and reports bootstrap confidence intervals over cases (Aubreville et al., 2022, arXiv version).

Threshold-free scores. F1 depends on the chosen operating point, so challenges also report precision–recall curves; AMIDA13 used them to show that the leaders’ advantage was not a matter of threshold choice. MIDOG 2022 added average precision, the mean precision at 101 recall values from 0 to 1, and the organizers found it a poor ranking metric. No approach’s curve reached zero precision, which they hypothesize results from detection thresholds applied before suppression; precision beyond the highest recall reached is commonly extrapolated as zero, which penalizes methods with high thresholds. In their results the runner-up, whose detection threshold was lower, scored a higher AP than the winner, whose precision–recall curve was clearly better (Aubreville et al., 2024, arXiv version).

Counts. Counting is scored by the mean absolute error of per-image counts (Lempitsky & Zisserman, 2010) or by the correlation between predicted and reference counts per region; AMIDA13 found good correlation even for some methods with lower object-level F1 (Veta et al., 2014, arXiv version), and the example shows why a count can hide errors that cancel.

Mask-based matching. The Cell Tracking Challenge matches a reference object to a computed one if the computed marker covers more than half of the reference marker, so each reference matches at most one computed object, and counts false negatives, false positives and the splits needed to separate clusters, weighted, as a detection cost (Matula et al., 2015). This needs object masks; for volumeless particles, Matula et al. note, no such test exists.

Applications

Cell detection supplies the counts and positions that bioimage analysis needs, such as nuclear counts and the positions of small spots, and the per-frame detections that tracking links, and nuclear positions and types for quantitative histology in digital pathology. Mitotic counts enter many tumour grading schemes, and challenge organizers present automatic detection as a way to reduce the effort and the inter-observer variation of counting by eye (Veta et al., 2014, arXiv version; Aubreville et al., 2022, arXiv version). The challenges evaluated detection on regions selected by pathologists; the MIDOG 2021 organizers note that whole-slide images contain out-of-focus, necrotic and other areas with hard negatives that the challenge did not test, and that predicting outcomes was outside its scope.

Open Problems

• The reference itself. Low object-level agreement between pathologists on mitoses has been reported in many studies (Aubreville et al., 2024, arXiv version), and different references change the scores: in MIDOG 2022, all methods lost recall against the PHH3-assisted reference, while the ranking barely changed. Against that reference, two experts scored F1 0.736 and 0.700 and the three-expert majority vote 0.761 (Aubreville et al., 2024, arXiv version).
• Generalization. Scanner, tissue and species shifts lowered performance in both MIDOG editions, by small but significant amounts in 2022, and the challenges did not test whole slides.
• History and protocol. Even the history is contested: Veta et al. (2014, arXiv version) trace automatic mitosis detection in breast cancer to work in the 1980s and 1990s, while Aubreville et al. (2024, arXiv version) date the first approaches to 2008. Radii, duplicate rules, pooling and AP definitions differ between benchmarks, so results are comparable only under the same protocol and code.