Bioimage Analysis

Bioimage analysis is the extraction of quantitative information from microscope images of biological samples: counting and outlining cells, following them over time, classifying their appearance and measuring their size, shape and the brightness of labelled molecules. It is also called bioimage informatics, the name Peng (2008) gave to an emerging area of bioinformatics that develops image processing, data mining, database and visualization techniques for molecular and cellular microscopic images, and image cytometry, the term Carpenter et al. (2006) used for automated cell image analysis. Computer vision supplies most of its methods, from thresholding and watershed to deep networks, but the goal differs from most vision applications: the output is a measurement that enters a biological argument, so how the measurement was produced matters as much as how accurate it looks.

This overview covers the imaging modalities in outline, the tasks, the open-source tools and their papers, deep learning and model sharing, file formats, FAIR data and image archives, the community’s benchmarks, and the reporting practices that make an analysis reproducible. Cell segmentation, the task most pipelines start with, has its own article.

Problem

Automated microscopes can image thousands of samples a day, in chemical and genetic screens and in time-lapse experiments, and Carpenter et al. (2006) placed the bottleneck at the analysis stage. A human observer typically scores one or a few features per cell, while image cytometry measures many at once: the intensity and location of each labelled component in each subcellular compartment, and the number, size and shape of those compartments (Carpenter et al., 2006). The Open Microscopy Environment’s founders made the same point from the instrument side: with digital detectors, the microscope became an assay device whose recorded signal, especially in fluorescence, is a quantitative measure of the number of target molecules (Goldberg et al., 2005).

The data make this hard. A genome-wide high-content screen may hold more than a million images, and light-sheet and slide-scanning instruments produce single images of gigapixels (Williams et al., 2017). Images come in hundreds of proprietary file formats (Moore et al., 2021), at scales from below a nanometre to centimetres (Sarkans et al., 2021), and the experimental metadata needed to interpret them are often not reported (Sarkans et al., 2021). Image processing invariably changes the numerical data (Schmied et al., 2024), and Aaron and Chew (2021) argue that incomplete reporting of image processing is pervasive and that the chosen details of a workflow can determine a study’s conclusions; the example below shows one such case.

History

Microscopy itself supplied the first milestones. Frits Zernike received the 1953 Nobel Prize in Physics for the phase contrast method and the phase contrast microscope, which makes thin, transparent cells visible by their phase shift (Thorn, 2016). Ernst Ruska built the first electron microscope that clearly outperformed the light microscope in 1933, work recognized with half of the 1986 Nobel Prize in Physics (Nobel press release, 1986). The 2008 Nobel Prize in Chemistry went to Osamu Shimomura, Martin Chalfie and Roger Tsien for the discovery and development of the green fluorescent protein (GFP); a fluorescent protein fused to a target protein lets it be imaged in live cells (Thorn, 2016). Huisken et al. (2004) introduced selective plane illumination microscopy (SPIM), a form of light-sheet microscopy, to image living embryos with optical sectioning and little photodamage. The 2014 Nobel Prize in Chemistry recognized Eric Betzig, Stefan Hell and William Moerner for using fluorescent molecules to bypass the resolution limit Ernst Abbe stipulated in 1873 (Nobel press release, 2014), and the 2017 prize recognized Jacques Dubochet, Joachim Frank and Richard Henderson for cryo-electron microscopy.

Software followed the arrival of digital detectors, which turned the microscope into a measuring device (Goldberg et al., 2005). ImageJ was first developed by Wayne Rasband in 1997 as a cross-platform version of NIH Image, and has been developed continuously since (ImageJ wiki); Schneider et al. (2012) recount 25 years of the two programs. As automated microscopes moved the bottleneck to analysis (Carpenter et al., 2006), CellProfiler (Carpenter et al., 2006), which its authors described as the first free, open-source system for flexible, high-throughput cell image analysis, let biologists chain modules into pipelines that ran over whole screens. The Open Microscopy Environment defined a data model for images, acquisition parameters and analysis results in XML (Goldberg et al., 2005). In 2012 Fiji, a distribution of ImageJ for biological image analysis, was published (Schindelin et al., 2012); a redesign of ImageJ itself, ImageJ2, had begun in 2010 (ImageJ wiki) and was described by Rueden et al. (2017). QuPath (Bankhead et al., 2017) brought the same open-source model to whole-slide pathology images, and ilastik, described in full by Berg et al. (2019), packaged interactive machine learning for users without substantial computational expertise.

Benchmarks gave the field a way to compare methods. The Broad Bioimage Benchmark Collection published annotated image sets for validating algorithms, the Cell Tracking Challenge launched at the IEEE International Symposium on Biomedical Imaging (ISBI) in 2013 (Maška et al., 2023), and a particle tracking competition, held as a workshop at ISBI 2012, compared methods on common data (Chenouard et al., 2014). Deep learning arrived through these benchmarks: U-Net was evaluated on the ISBI electron-microscopy and cell tracking challenges, and in the 2018 Data Science Bowl the top entries, all deep convolutional networks, segmented nuclei across experiments without the per-experiment configuration that classical pipelines needed (Caicedo et al., 2019). Content-aware image restoration (CARE; Weigert et al., 2018) extended learning to restoring images, and the BioImage Model Zoo (Ouyang et al., 2022) and ZeroCostDL4Mic (von Chamier et al., 2021) addressed how to share and train models.

As data grew, attention turned to keeping it usable. The FAIR principles (Wilkinson et al., 2016) asked that data be findable, accessible, interoperable and reusable; the Image Data Resource (Williams et al., 2017) and the BioImage Archive, established at EMBL-EBI in July 2019 (Sarkans et al., 2021), gave published images a home; OME-NGFF (Moore et al., 2021) proposed a cloud-ready file format; and community checklists for publishing images and image analyses followed (Schmied et al., 2024).

Visual Data

Light microscopy techniques divide broadly into brightfield and fluorescence (Thorn, 2016). The choice of modality sets what an image can show and what analysis it needs.

Brightfield and phase contrast

In brightfield microscopy, light passes through the sample to the objective, and the sample is seen by how it absorbs, scatters or deflects that light (Thorn, 2016). Most cells are thin and transparent, so they absorb little; phase contrast, in which cells appear dark on a light background, and differential interference contrast (DIC), which gives a shaded, pseudo-3D look, convert the phase shift that cells cause into visible contrast (Thorn, 2016). Stained tissue samples are also imaged in brightfield (Caicedo et al., 2019); histology analysis is the subject of digital pathology. Brightfield images need no labels, but their contrast does not single out a molecule of interest, as fluorescent labels do.

Fluorescence: widefield and confocal

Fluorescence microscopy images fluorophores, molecules that absorb light at one wavelength and emit at a longer one; labels are introduced either as fluorescently labelled antibodies (immunofluorescence, in fixed cells) or as genetically encoded fluorescent proteins fused to a target, which allows live imaging (Thorn, 2016). Thorn describes four colours per sample as straightforward, typically with 405, 488, 561 and 640 nm excitation, so a fluorescence image is a stack of channels, each showing one labelled structure, with a DNA stain such as DAPI often marking the nuclei.

Most cell biology imaging is widefield: the microscope forms an image of the sample on a camera (Thorn, 2016). The objective’s numerical aperture (NA) sets the resolution. Thorn gives the theoretical lateral and axial resolution as

$$d_{xy} = \frac{0.61\,\lambda}{\mathrm{NA}}, \qquad d_z = \frac{2\,\lambda\, n}{\mathrm{NA}^2},$$

where $\lambda$ is the wavelength of light, $n$ the refractive index of the sample and NA the sine of the largest collected angle times the refractive index the objective is designed for, and states the sampling rule that at least two camera pixels must span each resolvable element. Thorn calls these theoretical resolutions and names neither the resolution criterion nor which wavelength to use. As illustrative arithmetic of ours, taking the emission wavelength, green emission at $\lambda = 0.52\ \mu\mathrm{m}$ with an oil objective of NA 1.4 and $n \approx 1.5$ gives $d_{xy} \approx 0.23\ \mu\mathrm{m}$ and $d_z \approx 0.8\ \mu\mathrm{m}$: a 3D stack is typically several times coarser along the optical axis than across it, and pixels in the sample should be about $0.11\ \mu\mathrm{m}$ or smaller. Out-of-focus light is a widefield microscope’s main limitation, because excitation illuminates a cone through the sample and the camera cannot separate blur from in-focus signal (Thorn, 2016).

Confocal microscopy removes that light by illuminating one point at a time and detecting through a pinhole that blocks light from other planes; mirrors raster the spot across the sample, and the image is built point by point on a point detector (Thorn, 2016). Spinning-disk confocals scan many points at once onto a camera, trading some rejection of out-of-focus light in thick samples for speed and sensitivity; Thorn’s comparison table gives a guideline maximum sample thickness of 100 to 200 μm for laser-scanning confocals, against 20 μm for widefield. Total internal reflection fluorescence (TIRF) microscopy excites only the few hundred nanometres next to the coverslip (Thorn, 2016).

Light-sheet

Light-sheet microscopes illuminate the sample with a thin sheet orthogonal to the detection axis, so only the focal plane is excited; this removes out-of-focus light and reduces photobleaching and phototoxicity, and fusing views from two objectives can give 3D images with isotropic resolution (Thorn, 2016). SPIM was built to image samples up to a few millimetres in size, such as live medaka fish and Drosophila embryos (Huisken et al., 2004). For analysis, light-sheet experiments mean large volumes, often time-lapse, and views from several directions that must be registered and fused.

Super-resolution

Abbe’s limit held optical resolution at about half the wavelength of light, about 0.2 μm (Nobel press release, 2014). Stimulated emission depletion (STED) microscopy, developed by Hell in 2000, confines fluorescence to a nanometre-sized spot that is scanned across the sample; single-molecule microscopy, first used by Betzig in 2006, images a few switched-on molecules at a time over many frames and superimposes the images (Nobel press release, 2014). Thorn (2016) cites resolutions as fine as 20 nm laterally and 50 nm axially, usually with special sample preparation, microscopes or fluorophores. These optical techniques are distinct from computational super-resolution of single images.

Electron microscopy

Electron microscopes image with electron beams focused by magnetic lenses. In the transmission microscope that Ruska developed, the beam passes through a thin section; in the scanning electron microscope, a focused beam scans the surface and emitted secondary electrons form the image (Nobel press release, 1986). The same press release puts the resolution of a visible-light microscope at about 4,000 Å (0.4 μm) and practical electron-microscope resolution at about 1 Å. Two branches matter for analysis. Cryo-electron microscopy and tomography determine the structures of macromolecules and their cellular context, with raw images archived in EMPIAR since 2013; volume EM acquires serial images through volumes of resin-embedded, heavy-metal-stained cells and tissues, typically thicker than 250 nm (Sarkans et al., 2021). Segmentation targets in EM include cells, neurons and organelles (Archit et al., 2025).

What the data look like

Across modalities, a bioimage is a multidimensional array with up to five axes (time, channel and three spatial axes; NGFF specification 0.6), with physical pixel sizes that often differ between the lateral and axial directions, and often with a bit depth that must be mapped to 8 bits for display (Schmied et al., 2024). Weigert et al. (bioRxiv version) describe the acquisition trade-offs as a design space of resolution, speed, light exposure and imaging depth, bounded by the photon budget the sample tolerates: imaging faster or more gently means fewer photons and noisier images. The image noise article covers the photon (Poisson) noise that dominates at low light. Schmied et al. (2024) note that magnification alone does not determine pixel size, which also depends on sampling and binning, so analyses need the pixel size from the image metadata.

Vision Tasks Involved

Moen et al. (2019) organize deep learning for cellular images around four applications: image classification, image segmentation, object tracking and augmented microscopy. Bioimage pipelines usually combine several of these steps, followed by measurement.

• Segmentation. Cell and nucleus segmentation gives each cell its own mask, so that cells can be counted and measured one by one. The cell segmentation article covers classical threshold-and-watershed pipelines, learned methods such as StarDist and Cellpose, and the instance-matching metrics used to score them. Semantic segmentation covers tissue regions and, in EM, structures such as membranes and organelles.
• Detection and counting. Some questions need only one point per object, such as nuclei counts or the positions of small spots; the Cell Tracking Challenge scores detection separately from segmentation (Maška et al., 2023).
• Tracking. Following cells through time-lapse sequences, including divisions that create lineages, is the core of the Cell Tracking Challenge, and following particles inside cells, for the analysis of intracellular dynamic processes, is the subject of particle tracking benchmarks (Chenouard et al., 2014). Most Cell Tracking Challenge methods link a prior per-frame segmentation (Maška et al., 2023), as in tracking-by-detection; the multi-object tracking article covers the association step.
• Classification and phenotyping. Cells or whole images are assigned to classes or phenotypes; ilastik’s object classification and CellProfiler Analyst’s machine-learning classification are examples in the tools below.
• Measurement. Per-object features of intensity, location, size and shape are the output of most pipelines, from CellProfiler’s standard assays (cell count, size, per-cell protein levels) to complex morphology (Carpenter et al., 2006).
• Restoration. Denoising, deconvolution and learned restoration improve images before analysis or allow gentler acquisition (Weigert et al., 2018); image denoising covers the general methods.
• Registration and stitching. Views, tiles or time points of the same sample must be aligned before they can be combined, as when light-sheet views are fused (Thorn, 2016); these are forms of image registration, and Peng (2008) lists registration among the essential techniques of bioimage informatics and atlas building among its applications.

Tools and Platforms

The field relies heavily on open-source software. Schmied et al. (2024) name Fiji–ImageJ, CellProfiler, KNIME and Python libraries such as scikit-image among the open-source packages, alongside commercial packages such as ZEN, LAS X, NIS-Elements, Amira-Avizo, Imaris, Arivis Vision 4D and Huygens. The descriptions below come from each project’s paper and official pages, with versions as listed in October 2026; they are not a ranking.

ImageJ and Fiji. ImageJ is public-domain software for processing and analyzing scientific images, with many derivatives and variants, including ImageJ2 and Fiji (ImageJ wiki). Its recordable macro language and plugin architecture have drawn contributions from non-programmers, amateur programmers and professional developers alike (Rueden et al., 2017). ImageJ2 is a total redesign, with a data model for arbitrarily large N-dimensional images, that keeps backwards compatibility with the original, now called ImageJ 1.x (Rueden et al., 2017). Fiji is a distribution of ImageJ focused on biological-image analysis, which shares new algorithms with users as ImageJ plugins through an integrated update system (Schindelin et al., 2012); it bundles many plugins (Fiji project page) and both ImageJ 1.x and ImageJ2 (ImageJ wiki), is licensed under the GNU GPL on top of the BSD-licensed ImageJ2 core, and is maintained by groups at the University of Wisconsin–Madison, Human Technopole and the MPI-CBG (Fiji project page). The ImageJ 1.x release notes list version 1.54p (17 February 2025) as current.

CellProfiler. CellProfiler builds analyses as pipelines of modules that turn microscopy images into measurements, aimed at high-throughput experiments (Carpenter et al., 2006; Stirling et al., 2021). It was first written in MATLAB, rewritten in Python 2 in 2010, and ported to Python 3 for CellProfiler 4 (Stirling et al., 2021); the latest release on its official repository is 4.2.8 (27 September 2024). In the 2018 Data Science Bowl, the classical reference segmentations against which the entries were compared were made with CellProfiler pipelines built by an expert analyst (Caicedo et al., 2019).

ilastik. ilastik brings machine-learning-based analysis to users without computational expertise: users label a few pixels or objects with brush strokes, a nonlinear classifier on a set of predefined image features is retrained, and the result is shown at once for correction (Berg et al., 2019). It provides workflows for pixel classification, object classification, counting and tracking on data with up to five dimensions, and trained projects can be run from the command line on new data (Berg et al., 2019). By limiting itself to predefined features, ilastik trades some accuracy for training from very sparse labels on ordinary PCs, a compromise its authors contrast with deep learning’s need for much larger training sets (Berg et al., 2019). The current stable release is 1.4.2 (28 May 2026).

QuPath. QuPath was designed as open-source software for digital pathology and whole-slide image analysis, with tools for tumour identification, biomarker evaluation, batch processing and scripting (Bankhead et al., 2017). Its project page states that it is GPL-licensed and not intended for clinical, diagnostic or therapeutic purposes; the latest release is 0.7.0 (2 March 2026).

napari. napari is a fast, interactive viewer for multidimensional images in Python, which overlays points, polygons and segmentations on 2D, 3D and higher-dimensional arrays and lets users annotate and edit them; it is a fiscally sponsored project of NumFOCUS (napari documentation). Its plugins connect it to analysis tools, as for Segment Anything for Microscopy (Archit et al., 2025). The latest release is 0.10.0 (30 October 2026).

Others and the community. Bankhead et al. (2017) name Icy with Fiji and CellProfiler among the open-source tools of the ecosystem led by ImageJ. The ImageJ and CellProfiler teams started the Scientific Community Image Forum (image.sc), where software-related questions about image analysis, acquisition and data management are discussed in a software-independent manner (Rueden et al., 2019).

Deep Learning and Model Zoos

In segmentation, U-Net’s encoder–decoder with skip connections was introduced on biomedical challenges (the U-Net paper article covers its results), and instance methods such as StarDist and Cellpose use U-Net-style networks to separate touching cells, as the cell segmentation article describes. Schmied et al. (2024) list U-Net for cell segmentation, Noise2Void for image reconstruction, StarDist and Cellpose for instance segmentation, DeepProfiler for feature extraction and Piximi for image classification as examples of how quickly deep learning was adapted to microscopy.

Restoration is another major use. In the bioRxiv version of CARE, Weigert et al. report restoring images acquired with 60-fold fewer photons, reaching near-isotropic resolution from up to 10-fold axial undersampling, and resolving sub-diffraction tubular and granular structures at 20 times higher frame rates than the state of the art, on seven biological examples. Such networks predict structure learned from training data rather than measuring it, in our reading, and Schmied et al. (2024) warn that improper application of restoration methods (deconvolution, Noise2Void, CARE) can produce misleading images and advise that these steps may need to be indicated in the figure or its legend when they are critical for understanding the image.

Other projects address access and sharing. ZeroCostDL4Mic packages training and inference of networks for segmentation (U-Net, StarDist), object detection (YOLOv2), denoising (CARE, Noise2Void), super-resolution microscopy (Deep-STORM) and image-to-image translation as notebooks on the free cloud resources of Google Colab, with tools to evaluate each model’s performance (von Chamier et al., 2021). The BioImage Model Zoo addresses the problem that few pretrained models were interoperable between tools: it is a community resource where standardized pretrained models can be shared, tested and downloaded for adaptation or direct deployment in end-user tools such as ilastik, deepImageJ, QuPath, StarDist, ImJoy, ZeroCostDL4Mic and CSBDeep (Ouyang et al., 2022, abstract). Each model is described by a YAML file (rdf.yaml or bioimageio.yaml) that compatible software reads, according to the specification repository at release 0.5.14.3 (16 October 2026).

More recent work aims at generalist and foundation models. Generalist segmenters trained on many image types aim to work without retraining (the cell segmentation article covers Cellpose), and in a challenge held at NeurIPS the winning entry was a Transformer-based network (Ma et al., 2024, arXiv version). Segment Anything for Microscopy (μSAM), a tool for segmentation and tracking, fine-tunes the Segment Anything foundation model (SAM) into generalist models for light and electron microscopy and implements interactive and automatic segmentation in a napari plugin; its authors motivate it by the observation that pretrained models degrade on data unlike their training data and can only be improved by retraining (Archit et al., 2025).

Data: Formats, FAIR Principles and Archives

Formats. Microscope vendors write proprietary formats, and open-source libraries translate them on the fly or convert them to open formats (Moore et al., 2021); Bio-Formats is one, which the ilastik authors recommend, through Fiji, for data that ilastik cannot read directly (Berg et al., 2019). The OME Data Model, expressed in XML, describes images together with acquisition parameters, annotations and analysis results (Goldberg et al., 2005). OME-TIFF stores the image planes in one or more multi-page TIFF files and embeds a complete OME-XML metadata block in each file’s header, so the metadata survive even if some files are misplaced (OME-TIFF documentation).

OME-TIFF and HDF5 are ill-suited to uses that need very high and frequent levels of access, such as training AI models and publishing reference datasets in the cloud, which motivated OME’s next-generation file format, OME-NGFF, built on Zarr (Moore et al., 2021). It stores pixel data in chunks at several resolution levels (a pyramid) and keeps label images such as segmentation masks in the same structure as the image they describe, so that original and derived data stay together (Moore et al., 2021). Its implementation, OME-Zarr, is described with its tools and resources in a community paper (Moore et al., 2023). As of October 2026, the current released specification is version 0.6, dated 14 September 2026 in its version history: an image is a Zarr group (Zarr version 3) of arrays, one per resolution level, with 2 to 5 dimensions; time axes come before channel axes, which come before spatial axes; each axis can carry a physical unit; and each resolution level is mapped to physical coordinates by a scale, optionally followed by a translation, while version 0.6 adds named coordinate systems and further transformations between them (NGFF specification 0.6).

FAIR principles. The FAIR Guiding Principles ask that data and metadata be findable (with globally unique, persistent identifiers and rich, indexed metadata), accessible (retrievable by identifier through an open protocol, with metadata kept even after the data are gone), interoperable (using shared vocabularies and formal languages) and reusable (with a clear licence, detailed provenance and community standards), with emphasis on making data actionable by machines as well as people (Wilkinson et al., 2016). For images, REMBI (Recommended Metadata for Biological Images) proposes categories of metadata for the study, study component, biosample, specimen, image acquisition, image correlation, image data and analyzed data, and names computer-vision researchers, who need labelled images and segmentations to train algorithms, as one of at least three categories of users of archived images (Sarkans et al., 2021).

Archives. The Image Data Resource (IDR) publishes reference image datasets from published studies, including high-content screens, multidimensional microscopy and digital pathology, and links them to genetic and chemical databases and to phenotypes expressed in controlled ontologies (Williams et al., 2017; IDR page). The BioImage Archive accepts data from any imaging modality that are associated with a peer-reviewed publication or are of value beyond a single experiment, excluding patient-identifiable medical data, and provides storage for added-value resources such as EMPIAR and IDR (BioImage Archive page). The bioRxiv preprint of Hartley et al. (version posted 11 February 2022) reports more than 1.5 petabytes of image data across this ecosystem. Schmied et al. (2024) discourage statements that images are “available upon request” and describe depositing original images in a public repository with an open licence as the recommended level, and in a dedicated, searchable image database as the ideal.

Example

A common measurement is the mean area of nuclei, segmented by a global threshold. In this synthetic experiment, the treatment makes the nuclear stain 30% brighter but leaves nuclear size unchanged, so the true answer to “does the treatment change nuclear size?” is no. The images are blurred by a Gaussian point-spread function, as any microscope blurs them, and the same pipeline (threshold, connected components, removal of objects touching the border) is run with four thresholding choices: two fixed values, Otsu’s method on the pooled pixels of all images, and Otsu’s method on each image separately. Each image’s mean nuclear area is one observation, and the conditions are compared with Welch’s t-test, as a typical analysis would. The code runs with NumPy 2.3.5, SciPy 1.16.3 and scikit-image 0.25.2.

import numpy as np
from scipy import ndimage as ndi, stats
from skimage import filters, measure, segmentation

def make_image(rng, brightness, n=30, size=256, blur=2.0):
    """Synthetic nuclear stain: discs of known radius, blurred by the optics, plus noise."""
    truth = np.zeros((size, size), int)
    signal = np.zeros((size, size))
    yy, xx = np.mgrid[:size, :size]
    centres, k = [], 0
    while k < n:
        c = rng.uniform(15, size - 15, 2)
        r = rng.normal(8.0, 1.0)                    # same size distribution in both conditions
        if all(np.hypot(*(c - d)) > r + q + 4 for d, q in centres):
            centres.append((c, r)); k += 1
            disc = (yy - c[0])**2 + (xx - c[1])**2 <= r**2
            truth[disc] = k
            signal[disc] = brightness * rng.lognormal(0.0, 0.2)   # per-nucleus stain level
    img = ndi.gaussian_filter(signal, blur) + 0.1                 # point-spread blur + background
    img += rng.normal(0, 0.05, img.shape)                         # camera noise
    return img, truth

def mean_area(img, t):
    labels = measure.label(img > t)
    labels = segmentation.clear_border(labels)
    areas = [p.area for p in measure.regionprops(labels) if p.area >= 20]
    return np.mean(areas)

def experiment(seed, n_images=8):
    rng = np.random.default_rng(seed)
    data = {"control": [make_image(rng, 1.0) for _ in range(n_images)],
            "treated": [make_image(rng, 1.3) for _ in range(n_images)]}   # brighter stain, same size
    pooled = np.concatenate([im.ravel() for ims in data.values() for im, _ in ims])
    choices = {"fixed 0.3": lambda im: 0.3,
               "fixed 0.5": lambda im: 0.5,
               "Otsu, pooled": lambda im, t=filters.threshold_otsu(pooled): t,
               "Otsu, per image": filters.threshold_otsu}
    true = {c: np.mean([np.mean(np.bincount(tr.ravel())[1:]) for _, tr in ims])
            for c, ims in data.items()}
    rows = {}
    for name, thr in choices.items():
        a = {c: [mean_area(im, thr(im)) for im, _ in ims] for c, ims in data.items()}
        p = stats.ttest_ind(a["treated"], a["control"], equal_var=False).pvalue
        rows[name] = (np.mean(a["control"]), np.mean(a["treated"]), p)
    return true, rows

true, rows = experiment(seed=0)
print(f"true mean area (px): control {true['control']:.1f}, treated {true['treated']:.1f}")
print(f"{'threshold':16s}{'control':>9s}{'treated':>9s}{'ratio':>7s}{'p':>9s}")
for name, (c, t, p) in rows.items():
    print(f"{name:16s}{c:9.1f}{t:9.1f}{t / c:7.2f}{p:9.1e}")

print("\np-values (Welch t-test, 8 images per condition) for seeds 0-4")
table = {name: [] for name in rows}
for seed in range(5):
    for name, (_, _, p) in experiment(seed)[1].items():
        table[name].append(p)
for name, ps in table.items():
    print(f"{name:16s}" + "".join(f"{p:9.1e}" for p in ps))

true mean area (px): control 203.7, treated 203.1
threshold         control  treated  ratio        p
fixed 0.3           301.3    327.9   1.09  2.9e-02
fixed 0.5           217.1    243.5   1.12  1.5e-03
Otsu, pooled        195.8    224.6   1.15  5.5e-04
Otsu, per image     213.2    211.8   0.99  7.9e-01

p-values (Welch t-test, 8 images per condition) for seeds 0-4
fixed 0.3         2.9e-02  1.0e-03  2.2e-03  4.4e-02  3.5e-03
fixed 0.5         1.5e-03  2.2e-04  9.5e-05  1.0e-05  1.1e-05
Otsu, pooled      5.5e-04  1.4e-04  9.2e-05  8.6e-07  1.3e-05
Otsu, per image   7.9e-01  5.9e-01  2.8e-01  3.5e-01  9.2e-01

On seed 0, the three choices that apply one threshold to every image report treated nuclei 9 to 15% larger, and their p-values stay below 0.05 on all five seeds; the per-image Otsu threshold reports no difference on any seed, which is the truth. The cause is the blur: a nucleus’s edge is a ramp, not a step, and a brighter nucleus crosses any fixed level farther from its centre, so a fixed threshold turns a difference in brightness into a difference in area. Otsu’s threshold, computed on each image, rises with the image’s brightness and cancels most of the effect. The absolute areas also depend on the choice (about 300 pixels at a threshold of 0.3 against a true 204), so an area measured by thresholding is only comparable with areas measured the same way.

Per-image thresholding is not a safe default either. In our reasoning, if a treatment changed the number of cells, the background or the spread of brightness within an image, a threshold computed per image would move with it and could create or hide a difference just as easily. Schmied et al. (2024) require that the same adjustments and processing steps be applied when images are compared quantitatively; in our reading, both a fixed threshold and per-image Otsu satisfy that wording, which is why the requirement at the minimal level of their checklists to report key processing parameters matters here, where those parameters decide the conclusion. With 8 images per condition treated as independent observations, the test is also a simplification of a real experiment, in which replicates are separate samples or experiments, not images (the statistical significance article discusses what such tests do and do not show). Synthetic data with a known answer, as here, are one way to check whether a measurement responds to the quantity it is meant to measure.

Datasets and Benchmarks

Broad Bioimage Benchmark Collection (BBBC). A collection of freely downloadable microscopy image sets, each with a description of the biological application and some form of ground truth, organized by the Broad Institute’s Imaging Platform for developing and testing algorithms (BBBC page). The 2018 Data Science Bowl’s images are part of it (as BBBC038; see the cell segmentation article).

Cell Tracking Challenge. Running since 2013, it benchmarks segmentation, detection and tracking on real and simulated 2D and 3D time-lapse sequences from several light-microscopy modalities, with secret test annotations and organizer-verified submissions (Maška et al., 2023); the cell segmentation article describes its data and measures.

2018 Data Science Bowl. A Kaggle competition on nucleus segmentation whose scored test images came from experiments absent from the training data (Caicedo et al., 2019); the cell segmentation article describes its data, protocol and results.

Particle tracking challenge. An open competition in which teams applied their methods to a commonly defined dataset covering diverse scenarios, scored with common measures; no single method performed best in every scenario, but clear differences between approaches emerged (Chenouard et al., 2014, abstract).

Multimodality cell segmentation challenge. Organized at NeurIPS to test whether one model can segment cells in brightfield, fluorescence, phase contrast and DIC images without manual adjustment (Ma et al., 2024, arXiv version). In that version, participants received 1,000 labelled and 1,725 unlabelled images and a 101-image tuning set; 28 teams submitted Docker containers that the organizers ran on a hidden test set of 422 images, all from new biological experiments and including two whole-slide images, and ranked by an F1 score at IoU 0.5 together with running time. The winning Transformer-based algorithm reached a median F1 of 89.7% on the test set.

Challenges

• Heterogeneity. Modalities, labels, magnifications and sample types differ between experiments, and a classical pipeline must be reconfigured for each (Caicedo et al., 2019). Pretrained models degrade on data unlike their training sets (Archit et al., 2025).
• The photon budget. Speed, resolution, depth and light exposure cannot all be maximized, because the sample tolerates only so much light (Weigert et al., bioRxiv version). Live imaging is therefore often noisy, and restoration methods that compensate bring their own risks.
• Anisotropy and dimensionality. Axial resolution is coarser than lateral resolution in widefield and confocal 3D microscopy (Thorn, 2016), and data have up to five dimensions; in our reading, methods written for 2D natural images need adapting to both.
• Scale. Screens with more than a million images and gigapixel light-sheet or slide images (Williams et al., 2017) call for batch processing and for formats that can be read in chunks from cloud storage (Moore et al., 2021).
• Annotation. Correct interpretation of biological images typically needs expert annotators, which makes manual annotation hard to scale (Berg et al., 2019); interactive tools such as ilastik and μSAM aim to reduce the effort, and image annotation covers the general practice.
• Metadata. Pixel sizes, channels and acquisition settings are needed to interpret measurements, yet metadata on imaging methods are reported to be vastly under-reported in biomedical research (Sarkans et al., 2021).

Evaluation

Segmentation and tracking are scored against annotations with object-level metrics, such as matching by IoU and counting true and false positives, and with benchmark-specific measures such as the Cell Tracking Challenge’s; the cell segmentation article explains these and how matching rules and averaging change the numbers. Challenges control the protocol by keeping test labels secret and running or verifying submitted software themselves (Maška et al., 2023; Ma et al., 2024, arXiv version).

For measurement, accuracy of the masks is not the whole story. The question is whether the extracted quantity tracks the biological variable without being driven by imaging conditions, as the example shows; synthetic images with known values, images of reference samples and comparison with an independent assay are ways to check it, a point that follows from the example rather than from a cited benchmark. For machine-learning workflows, Schmied et al. (2024) require at the minimal level that validation data not be part of the training and testing data, and recommend that authors discuss and ideally test how well the model performed on their data and show its limitations.

Reproducibility and reporting

The community checklists of Schmied et al. (2024), developed by the Image Analysis and Visualization Working Group of the QUAREP-LiMi initiative, define three reporting levels, minimal, recommended and ideal, for image figures and for image-analysis workflows. They separate three kinds of workflows:

• Established workflows, such as published CellProfiler pipelines or ImageJ plugins: authors must cite the workflow, report key processing parameters, provide example input and output data, document manual interventions such as hand-drawn regions of interest, and give exact software versions. Reporting all settings and sharing examples in a public repository is recommended, and screen recordings or containers are ideal.
• New workflows, assembled from existing components: authors must also describe the sequence of steps, report settings that deviate from defaults and share the workflow as code or a pipeline file; describing the rationale and limitations is recommended.
• Machine-learning workflows: authors must cite the original method, give access to the trained model and provide validation data; sharing the full training and testing data and the training code is recommended, and standardized model formats such as the BioImage Model Zoo’s are named as the ideal.

For image figures, the same checklists ask for scale bars, annotated channels, the same contrast adjustments for images that are compared, colour schemes visible to colour-blind readers, and caution with non-linear adjustments such as gamma and histogram equalization (Schmied et al., 2024). Aaron and Chew (2021) argue that proper reporting of image processing improves reproducibility and strengthens the conclusions drawn from the results, and Jost and Waters (2019) review how to validate sample preparation and imaging and how to avoid bias in the acquisition and analysis of images.

Failure Modes and Risks

• Undocumented processing choices. A threshold, filter or normalization changes measurements and can decide a conclusion, as in the example; if it is not reported, readers cannot judge or repeat the result (Aaron and Chew, 2021).
• Imaging conditions read as biology. In our reading, differences in brightness, focus or staining between conditions can propagate through segmentation into areas, counts and intensities; the example’s fixed threshold converts a stain difference into a size difference.
• Bias in acquisition and analysis. Microscopes and preparation methods impart artifacts to images, and observers interpret images with an unconscious bias toward what fits their current understanding; together these can lead to false conclusions (Jost and Waters, 2019, abstract).
• Restoration artifacts. Restoration methods such as deconvolution, Noise2Void and CARE can give misleading images when applied improperly (Schmied et al., 2024).
• Biased training data. Biases in training data and errors in ground-truth labels bias supervised models (Schmied et al., 2024), and pretrained models degrade on data unlike their training data (Archit et al., 2025).
• Lost data and software. Sharing images “upon request” has been shown to be inefficient, and analysis software needs maintenance to stay usable (Schmied et al., 2024).

Deployment Considerations

Analyses at scale run as batch workflows: CellProfiler pipelines over whole screens, ilastik projects from the command line after interactive training (Berg et al., 2019), and scripted Fiji or Python workflows. Exact versions should be recorded, and containers such as Docker or Singularity avoid variation between computers and operating systems (Schmied et al., 2024); the multimodality challenge required Docker containers to standardize its evaluation (Ma et al., 2024, arXiv version). Chunked, multiscale formats such as OME-Zarr let viewers and analysis code read parts of large datasets directly from object storage (Moore et al., 2021). Models are shared in the BioImage Model Zoo’s format so that one trained network runs in several tools, and free cloud notebooks lower the hardware barrier for training (von Chamier et al., 2021). Finally, depositing original images, analysis code and trained models in public repositories, with REMBI-style metadata, turns a one-off analysis into data that others can check and reuse (Sarkans et al., 2021; Schmied et al., 2024).