Digital Pathology

Digital pathology is the practice of scanning glass microscope slides of tissue into images and viewing, managing and analyzing them on a computer. The images, called whole-slide images (WSIs) or virtual slides, are among the largest in medical imaging: a single slide scanned at the resolution pathologists use for diagnosis is billions of pixels. Applying computer vision and machine learning to these images is often called computational pathology. It covers measuring stains, finding and classifying cells and tissue regions, and predicting a label for a whole slide or patient from its image, for research and as decision support for pathologists.

The field combines several problems that are rare elsewhere in vision: images too large to load, color that varies with how a laboratory stained the tissue, labels that usually exist only for the whole slide, and evaluation that must say which reference standard and which readers a method was compared with. This article surveys these problems and links to the articles that treat the underlying methods. It describes methods, evidence and limits for practitioners and researchers, gives no diagnostic guidance, and attributes reported results to their authors with the study design behind them.

Problem

A pathologist examines thin sections of tissue removed by biopsy or surgery, stained so that cells and structures become visible under a light microscope. Much of this work is visual search and grading. In breast cancer staging, for example, pathologists examine sentinel lymph node sections for metastases; Ehteshami Bejnordi et al. (2017) describe this assessment as tedious and time-consuming, and cite a retrospective study in which expert review changed the nodal status of 24% of patients. In prostate biopsies, Gleason grading assigns growth patterns that Bulten et al. (2022) call inherently subjective, with considerable variability between and within pathologists.

Once slides are digitized, a computer can do some of this search, quantify what the eye only estimates (the fraction of cells that express a protein, for example), and help in tasks where readers disagree. In the FDA’s final order classifying the device type, a whole-slide imaging system is identified as an automated system for creating, viewing and managing digital slides, intended as an aid to the pathologist to review and interpret digital images of surgical pathology slides (Food and Drug Administration, 2018). Analysis software is a separate device type (Food and Drug Administration, 2023), described below under regulation.

History

DICOM Supplement 145, the 2010 extension of the medical imaging standard to whole-slide images, describes pathology as undergoing a transformation driven by the commercial availability of instruments that digitize microscope slides, and notes that DICOM had until then no way to handle images of this size, tiled access or multiple resolutions (NEMA, 2010).

Machine-learning benchmarks on whole slides followed. The organizers of CAMELYON16, a challenge to detect breast cancer metastases in lymph node sections, describe it as the first challenge using whole-slide images (CAMELYON17 challenge pages); its results, published in JAMA, compared the submitted algorithms with pathologists (Ehteshami Bejnordi et al., 2017). Two shifts came next. Campanella et al. (2019) presented a system trained with only the diagnoses in pathology reports as labels and evaluated it on 44,732 slides, avoiding pixel-level annotation and training at what they call unprecedented scale. From 2023, several groups pretrained large encoders, most of them self-supervised on 100,000 to more than a million slides and one on image–caption pairs, the pathology foundation models described below.

Regulation moved in parallel in the United States. The FDA announced the availability of guidance on the technical performance assessment of whole-slide imaging devices in April 2016, created the device type “whole slide imaging system” in April 2017, and the type “software algorithm device to assist users in digital pathology” in September 2021, each through a De Novo classification (Food and Drug Administration, 2016; 2018; 2023).

Visual Data

Slides and stains

Tissue removed by biopsy or resection is preserved, processed, cut into thin sections, mounted on glass slides and stained. Each step is a source of variation that analysis must survive: the FDA’s special controls for analysis software name fixation type and time and histology slide processing techniques among the pre-analytical variables a training set must represent (Food and Drug Administration, 2023). Tissue may be formalin-fixed and paraffin-embedded (FFPE) or frozen, and public collections such as The Cancer Genome Atlas contain slides of both kinds (Chen et al., 2024, arXiv version).

Hematoxylin and eosin (H&E) is the routine stain. In Macenko et al.’s (2009) description, hematoxylin stains nucleic acids blue-purple and eosin stains proteins pink; Tellez et al. (2019) describe H&E as the most widely used staining worldwide, highlighting nuclei in blue and cytoplasm, connective tissue and muscle in shades of pink. Immunohistochemistry (IHC) reveals a chosen protein with a colored chromogen, such as the brown diaminobenzidine (DAB), usually over a blue hematoxylin counterstain. The image bundled with scikit-image and used below shows colonic glands in which DAB reveals the protein FHL2, with hematoxylin counterstaining (scikit-image source). When building CAMELYON16’s reference standard, IHC with an anti-cytokeratin antibody was used for all difficult cases and all cases that appeared negative on H&E, which the authors say minimizes false-negative interpretations (Ehteshami Bejnordi et al., 2017).

Most stains only absorb light. Slides are viewed with light passing through them from below, so unstained areas appear white and stained areas darker, by an amount that depends on the stain and its quantity (Macenko et al., 2009). Color in pathology images is therefore modeled as absorbance, the basis of color deconvolution below.

Scanners and whole-slide images

A slide scanner digitizes the tissue area of a slide at microscopic resolution into one image. Resolution is quoted in micrometers per pixel (mpp). DICOM Supplement 145 records the convention that 0.25 mpp is called 40× and 0.5 mpp 20×, by analogy with a microscope’s 40× objective, although scanners use no eyepiece and may not use microscope objectives (NEMA, 2010). The same document’s typical example, a 20 mm × 15 mm sample at 0.25 mpp, gives an image of about 80,000 × 60,000 pixels (4.8 gigapixels), about 15 GB as uncompressed 24-bit color. Thick samples may be scanned at several focal planes (z-planes), and multispectral scanners record more bands (NEMA, 2010).

Scanners in published studies illustrate the range. CAMELYON16’s slides came from a 3DHISTECH Pannoramic 250 Flash II with a 20× objective (0.243 µm pixels) at one center and a Hamamatsu NanoZoomer-XR with a 40× objective (0.226 µm) at the other (Ehteshami Bejnordi et al., 2017). PANDA’s biopsies were scanned on four scanner models from 3DHISTECH, Hamamatsu Photonics and Leica Biosystems (Bulten et al., 2022), and Virchow’s 1.5 million training slides on Leica scanners at 20× (0.5 mpp) (Vorontsov et al., 2024). These are the devices the studies report, not a survey or ranking of the market.

Pyramidal formats

Pathologists pan across a slide at low magnification and zoom into regions of interest, and Supplement 145 asks that the organization of the stored data support this access pattern (NEMA, 2010). Two ideas meet it:

• Tiling. The image is stored as square tiles rather than full rows, so that reading a region loads only the tiles that cover it. The supplement gives typical tile sizes of 240 × 240 to 4,096 × 4,096 pixels.
• A resolution pyramid. Lower-resolution copies are precomputed so that a zoomed-out view does not require reading and downsampling the whole base image. With levels a factor of 2 apart the supplement estimates about 32% extra storage, and with a factor of 4 about 7%. This is the image pyramid of image processing, stored on disk.

In DICOM, each pyramid level is stored in one or more multi-frame image objects whose frames are the tiles, and the levels form a series that may also hold a thumbnail and ancillary images such as the slide label and a macro image of the whole slide (NEMA, 2010). Tiling keeps each frame within the 16-bit row and column limit of earlier DICOM images, while a Total Pixel Matrix of up to $(2^{32})^2$ pixels places the tiles on the slide. The current standard distinguishes two tile layouts: TILED_FULL, in which the frames cover the whole rectangle without gaps or overlap in an implied order (row by row, then focal plane, then optical path), and TILED_SPARSE, in which each frame’s position is stated explicitly and tiles may be missing or overlap (NEMA, 2026). Sparse tiling exists because some scanners do not capture every region at full resolution (NEMA, 2010).

Supplement 145 also notes that WSIs are often stored compressed, with JPEG or JPEG 2000, and reports that pathologists had found no loss of diagnostic information at JPEG ratios of 15–20× or JPEG 2000 ratios of 30–50× for most applications (NEMA, 2010). That is the committee’s 2010 summary of the field, not a measurement in the supplement.

Before DICOM, and alongside it, each scanner vendor defined its own format. The OpenSlide library, which reads many of them through one C interface, lists Aperio (.svs), Hamamatsu (.vms, .vmu, .ndpi), Leica (.scn), MIRAX (.mrxs), Philips (.tiff), Ventana (.bif, .tif), Zeiss (.czi) and others, many of them pyramidal tiled TIFF or BigTIFF variants with non-standard metadata, together with DICOM and generic tiled TIFF (OpenSlide, version 4.0.1, as of October 2026). The project states that its support is not endorsed by the vendors and may be incomplete. CAMELYON’s slides, for comparison, are distributed as multi-resolution TIFF files with level 0 as the highest resolution (CAMELYON17 data page).

Working at Gigapixel Scale

A whole slide is far larger than any network’s input image, so the methods below share a tile-based pipeline:

1. Find the tissue. A large part of a slide is not covered by tissue, so background is removed first: in a 2018 arXiv preprint by Campanella, Werneck Krauss Silva and Fuchs, cited below as Campanella et al. (2018, arXiv) and distinct in title and author list from the 2019 Nature Medicine study, background tiles are discarded before training; CLAM’s pipeline segments the tissue automatically before cutting patches (Lu et al., 2021, arXiv version).
2. Cut tiles (patches) at a chosen magnification, for example 256 × 256 pixels in CLAM. In Campanella et al.’s prostate needle biopsies, a slide gave about 100 non-overlapping tissue tiles at 5× and 1,000 at 20× (2018, arXiv); other slides give far more, as many as 70,121 tiles in the data of Xu et al. (2024).
3. Process each tile with a network: a classifier that outputs a probability, a segmentation network, or an encoder that outputs a feature vector.
4. Aggregate the tile outputs into a heatmap, a list of detections, or a slide-level prediction.

Magnification trades context against detail, and Campanella et al. (2018, arXiv) compared 5×, 10× and 20× for their task. Computing features once and training only the aggregation on them, as CLAM does, reduces the data, in its authors’ estimate, nearly 200-fold, so that training on thousands of slides fits on a workstation (Lu et al., 2021, arXiv version). Veeling et al. (2018) start from the observation that histopathology images are symmetric under rotation and reflection, and build patch classifiers equivariant to 90° rotations and reflections.

Stain Variation and Normalization

The same tissue stained in two laboratories looks different. Macenko et al. (2009) attribute the variation to the stain manufacturer, storage before use, the staining procedure and fading after staining; Vahadane et al. (2016) add the color responses of the scanners. Networks trained on one laboratory’s slides often underperform on another’s (Tellez et al., 2019), so stain variation is a central form of dataset bias in pathology.

Color deconvolution

The physical model is the Beer–Lambert law. With intensities $I_c$ normalized to $[0, 1]$ by the brightness of empty glass, the optical density of channel $c$ is

$$\mathrm{OD}_c = -\log_{10} I_c ,$$

and a mixture of stains gives a linear combination of their optical densities (Macenko et al., 2009, Eq. 1, following standard practice):

$$\mathbf{OD} = V\,\mathbf{s} \quad\Longrightarrow\quad \mathbf{s} = V^{-1}\,\mathbf{OD},$$

where each column of the $3 \times 3$ matrix $V$ is one stain’s OD vector (its absorbance in red, green and blue) and $\mathbf{s}$ holds the amounts of the stains at the pixel (Macenko et al., Eq. 2). Ruifrok and Johnston (2001) introduced this color deconvolution for quantifying up to three stains imaged with an RGB camera, testing it on combinations of DAB, hematoxylin and eosin. Converting to optical density is what makes the model linear: in RGB the pixels of two stains are separable only along a curve, in OD space by a straight line through the origin (Macenko et al., 2009, Fig. 2). The color spaces designed for natural images, such as HSV or CIELAB, do not model absorbance in this way.

Normalization and augmentation

Color deconvolution needs stain vectors. A fixed standard matrix ignores the variation between slides, and measuring vectors from a region stained with one dye is tedious (Macenko et al., 2009). Normalization methods estimate the vectors or the color distribution from each image and map it to a reference:

• Macenko et al. (2009) estimate the two stain vectors of an H&E image automatically. Pixels are converted to OD, nearly unstained pixels (OD below $\beta = 0.15$) are dropped, and the remaining OD vectors are projected onto the plane of the two leading singular vectors. The angle of each projected pixel is computed, and the robust extremes of the angles (the $\alpha$-th and $(100-\alpha)$-th percentiles, with $\alpha = 1$) give the two stain directions. A second step rescales each stain’s intensity so that its 99th percentile matches a reference.
• Vahadane et al. (2016) decompose an image into stain density maps constrained to be non-negative and sparse, on the observation that most tissue regions are dominated by one stain, and combine an image’s density maps with the stain colors of a target image, which changes its color while preserving its structure (abstract).
• Reinhard et al. (2001) proposed color transfer between natural images in computer graphics; Tellez et al. (2019) list it with Macenko et al. among the traditional approaches to stain normalization. Later methods use nucleus detection, generative models or style transfer (Tellez et al., 2019).

The alternative is to make the network insensitive to color by stain augmentation: random perturbations in HSV or in the H&E-DAB stain space during training. Tellez et al. (2019) compared both on H&E patches from four organs and nine laboratories and concluded that color augmentation should always be used, that it can be combined with a network-based normalization for more robust results, and that omitting normalization costs little performance where computation is limited.

Example: separating stains

The example separates the bundled IHC image into stain channels with scikit-image’s implementation of color deconvolution, then re-renders the same tissue as two other laboratories might have stained it. It runs with NumPy 2.3.5 and scikit-image 0.25.2; the image (skimage.data.immunohistochemistry, a 512 × 512 RGB image acquired at the Center for Microscopy And Molecular Imaging, described as having no known copyright restrictions) is a file inside the package, so nothing is downloaded. Two details of the implementation matter for reading its output, both from its source. Its matrix rgb_from_hed has hematoxylin, eosin and DAB vectors as rows, which the source attributes to Ruifrok and Johnston (2001). And separate_stains floors intensities at $10^{-6}$, divides their natural logarithm by $\ln 10^{-6}$, unmixes with the inverse matrix and clips negative amounts to zero. Since $\ln I / \ln 10^{-6} = -\log_{10} I / 6$, its stain amounts are those obtained by unmixing base-10 OD divided by 6; the script checks this.

import numpy as np
from skimage import data
from skimage.color import rgb2hed, combine_stains, rgb_from_hed, hed_from_rgb
from skimage.filters import threshold_otsu
from skimage.util import img_as_ubyte

ihc = data.immunohistochemistry()            # 512x512 RGB, DAB + hematoxylin
hed = rgb2hed(ihc)                           # stain amounts: H, E, DAB
print("stain vectors (rows H, E, DAB):\n", rgb_from_hed)

# scikit-image's units: base-10 optical density / 6, unmixed by the inverse matrix
od = -np.log10(np.maximum(ihc / 255.0, 1e-6))
print("max |OD/6 @ inv(M) - rgb2hed|:",
      np.abs(np.maximum(od @ hed_from_rgb / 6, 0) - hed).max())
for i, name in enumerate(["H", "E", "DAB"]):
    print(f"{name:>3}: median {np.median(hed[..., i]):.4f}, "
          f"99th pct {np.percentile(hed[..., i], 99):.4f}")

# Treat these amounts as ground truth and re-render the tissue as other labs would
def render(amounts, M):
    return img_as_ubyte(combine_stains(amounts, M))

def unit(v):
    v = np.asarray(v, float)
    return v / np.linalg.norm(v)

lab_a = render(hed, rgb_from_hed)
t = threshold_otsu(rgb2hed(lab_a)[..., 2])   # one DAB threshold for all labs
truth = hed[..., 2] > t

darker = hed.copy()
darker[..., 0] *= 1.6                        # lab B: 60% more hematoxylin
lab_b = render(darker, rgb_from_hed)
M_c = rgb_from_hed.copy()
M_c[0] = unit([0.50, 0.80, 0.33])            # lab C: a bluer hematoxylin
lab_c = render(hed, M_c)
angle = np.degrees(np.arccos(unit(rgb_from_hed[0]) @ M_c[0]))
print(f"lab C hematoxylin vector is {angle:.1f} degrees from the standard one")

def rgb_rule(img):                           # a hand-set "brown pixel" colour rule
    r, b = img[..., 0].astype(int), img[..., 2].astype(int)
    return (r - b) > 20

print("lab  mean RGB            DAB+ (deconv)  agree w/ truth  DAB+ (RGB rule)")
for name, img in [("A", lab_a), ("B", lab_b), ("C", lab_c)]:
    dab = rgb2hed(img)[..., 2] > t           # always the standard matrix
    print(f" {name}   {np.round(img.reshape(-1, 3).mean(0), 1)}   "
          f"{dab.mean():.3f}          {(dab == truth).mean():.3f}"
          f"           {rgb_rule(img).mean():.3f}")

stain vectors (rows H, E, DAB):
 [[0.65 0.7  0.29]
 [0.07 0.99 0.11]
 [0.27 0.57 0.78]]
max |OD/6 @ inv(M) - rgb2hed|: 2.220446049250313e-16
  H: median 0.0166, 99th pct 0.0607
  E: median 0.0000, 99th pct 0.0000
DAB: median 0.0539, 99th pct 0.1918
lab C hematoxylin vector is 10.6 degrees from the standard one
lab  mean RGB            DAB+ (deconv)  agree w/ truth  DAB+ (RGB rule)
 A   [175.7 143.9 142.4]   0.469          0.999           0.586
 B   [159.7 130.  136.2]   0.469          0.999           0.514
 C   [182.6 140.4 140.9]   0.487          0.982           0.627

The eosin channel is empty, as expected for a slide stained only with DAB and hematoxylin: the small negative amounts that the inverse matrix assigns to eosin are clipped to zero. The re-rendered images give the test a known answer, since every pixel’s stain amounts are fixed and only the staining changes. Lab A uses the same matrix, and the 0.1% of pixels that disagree come from rounding to 8 bits.

Lab B’s extra hematoxylin darkens the image by about 16 gray levels in red and 14 in green, yet the deconvolved DAB-positive fraction is unchanged. In OD space a change in one stain’s amount moves pixels along that stain’s own vector, and the inverse matrix assigns the change to the hematoxylin channel alone. The hand-set RGB rule, which mixes both stains in one color difference, drops by 7 percentage points. Lab C is the harder case: a hematoxylin with a different hue violates the assumption that the matrix is right, so part of the hematoxylin leaks into the DAB channel and 1.8% of pixels change class. Per-slide estimation of stain vectors, as in Macenko et al.’s and Vahadane et al.’s methods, addresses this second kind of variation; in this synthetic setting the error is small because only one vector moved, by about 11°.

Vision Tasks and Methods

Nucleus segmentation

Nucleus segmentation is an instance segmentation problem with thousands of touching, irregular instances per tile; the cell segmentation article covers its classical pipelines, the learned families (boundary classes, star-convex polygons, flow fields, distance maps) and HoVer-Net’s joint segmentation and classification of nuclei in histology, with the instance-matching metrics and histology datasets used to compare them. Because hematoxylin marks the nuclei, it follows that the hematoxylin channel from color deconvolution is a convenient input for classical methods on H&E images.

Tissue classification and detection

A basic learned task classifies tiles: tumor or normal, or one of several tissue types. Run over a slide, a tile classifier produces a probability heatmap. In CAMELYON16, 25 of the 32 submitted algorithms were based on convolutional networks, and the best slide classifier used a GoogLeNet architecture (Ehteshami Bejnordi et al., 2017). Detection tasks report each lesion’s location; CAMELYON16’s first task scored lists of metastasis locations against the reference annotations.

Tile classifiers need tile labels, which come from region annotations drawn by or under the supervision of pathologists. CAMELYON16’s metastases were outlined by two students and checked by one of two pathologists (Ehteshami Bejnordi et al., 2017). Such annotations are expensive; Campanella et al. (2018, arXiv) note that studies had relied on datasets of a few hundred slides, which motivated the weakly supervised methods below. Patch datasets such as PatchCamelyon, derived from CAMELYON16 slides, turn this task into a standard image classification benchmark (Veeling et al., 2018).

Slide-level prediction by multiple-instance learning

Many labels in pathology belong to a slide, a specimen or a patient: the diagnosis in the report, a grade, a mutation found by sequencing. Multiple-instance learning (MIL) treats a slide as a bag of tile instances with one bag label. Under the standard assumption, as Ilse et al. (2018) state it for a setting they trace to Dietterich and colleagues in 1997, a bag is positive if and only if at least one instance is positive:

$$Y = \max_k \, y_k ,$$

where $y_k \in \{0, 1\}$ is the unknown label of tile $k$ and $Y$ the slide label. The assumption fits detection tasks, in which a single small metastasis makes a node positive. Lu et al. (2021, arXiv version) point out that it suits neither multi-class problems nor binary ones without an intrinsic positive and negative class.

A MIL model must be invariant to the order of the tiles, so it combines tile outputs with a symmetric pooling function. Ilse et al. distinguish pooling tile scores (the instance-level approach) from pooling tile embeddings and classifying the result (the embedding-level approach), and note that the mean is a poor operator for tile scores although it can work on embeddings.

• Max pooling of tile scores. Campanella et al.’s preprint (2018, arXiv v2; a different title and author list from the 2019 study) describes training under the MIL assumption: in each epoch every tile of every slide is scored, and the network is trained on the top-ranked tile of each slide with the slide’s label. At test time a slide is positive if any tile exceeds a threshold. The published study (Campanella et al., 2019) reports a MIL-based system trained with only the reported diagnoses as labels on 44,732 slides from 15,187 patients without data curation, with areas under the ROC curve above 0.98 on prostate cancer, basal cell carcinoma and breast cancer metastases to axillary lymph nodes, as its abstract reports; the abstract, the only part read for this article, does not describe the test sets or their reference standard.
• Attention pooling of embeddings. Ilse et al. (2018) replace fixed pooling with a learned weighted average. For tile embeddings $\mathbf{h}_k$, the gated attention weights are

$$a_k = \frac{\exp\{\mathbf{w}^\top(\tanh(V\mathbf{h}_k) \odot \operatorname{sigm}(U\mathbf{h}_k))\}}{\sum_j \exp\{\mathbf{w}^\top(\tanh(V\mathbf{h}_j) \odot \operatorname{sigm}(U\mathbf{h}_j))\}}, \qquad \mathbf{z} = \sum_k a_k \mathbf{h}_k ,$$

and a classifier on the slide representation $\mathbf{z}$ gives the prediction; the weights $a_k$ show which tiles drove it.

• CLAM (Lu et al., 2021) builds on this pooling with one attention branch per class and an auxiliary clustering loss: in each slide the most and least attended tiles receive pseudo-labels as positive and negative evidence, which constrains the tile feature space (arXiv version). Its tiles are encoded once by a ResNet-50 pretrained on ImageNet, truncated after the third residual block, giving 1,024-dimensional features. The authors report 10-fold Monte Carlo cross-validated test AUCs of 0.991 for three-class renal cell carcinoma subtyping, 0.956 for lung cancer subtyping and 0.953 for lymph node metastasis detection on combined CAMELYON16 and CAMELYON17 slides, all on public data. The caption of their Figure 2 reports CLAM ahead of a max-pooling MIL baseline (a multi-class variant for renal subtyping) for nearly all training-set sizes in all three tasks, while the main text says it outperforms the baseline for all tasks and training-set sizes. They caution against reading attention heatmaps as pixel-accurate segmentation masks, since the scores only rank regions relative to each other within a slide.

PANDA’s leading teams used a related design trained end to end: sample patches from a slide, encode them with a convolutional network, concatenate the features and classify the slide’s grade group directly (Bulten et al., 2022).

Example: pooling tile scores into a slide prediction

How well a pooling rule works depends on how much of a positive slide is tumor and on what the classifier does on normal tissue. The script below makes this concrete with synthetic tile probabilities, standing in for the output of an imperfect tile classifier on 2,000 tissue tiles per slide. The generator is our own and makes four assumptions, each stated in the code: normal tiles score low, and their distribution may shift from slide to slide (standing in for differences in staining and tissue; the code compares no shift with a shift whose size, a standard deviation of 0.5 in the log of the beta distribution’s second parameter, is our choice, not a measured value); a few artifact tiles per slide score high (folds, ink, blur; Campanella et al.’s preprint reports that top-ranked tiles in negative slides included edges and inked regions); tumor tiles score mostly high; and the tumor fraction ranges from one tile to 20% of the slide. It compares max pooling, mean pooling and the mean of the ten highest scores, with no learned attention, by the slide-level ROC AUC over 200 negative and 200 positive slides, with and without the shift and with and without artifacts. Each cell gives the mean and range over seeds 0–4. It runs in a few seconds with NumPy 2.3.5 and scikit-learn 1.7.2.

import numpy as np
from sklearn.metrics import roc_auc_score

N_PATCHES = 2000                                   # tissue patches per slide

def patch_scores(rng, tumour_frac, artefacts_per_slide, shift):
    """Tumour probabilities that an imperfect patch classifier gives one slide."""
    k = 12 * np.exp(rng.normal(0, shift))          # slide-to-slide shift in the
    s = rng.beta(1, k, N_PATCHES)                  # scores of normal tissue
    n_art = rng.poisson(artefacts_per_slide)       # folds, ink, blur: false alarms
    s[rng.choice(N_PATCHES, n_art, replace=False)] = rng.beta(6, 2, n_art)
    n_tum = int(round(tumour_frac * N_PATCHES))
    s[rng.choice(N_PATCHES, n_tum, replace=False)] = rng.beta(5, 2, n_tum)
    return s

pools = {"max": np.max,
         "mean": np.mean,
         "top-10 mean": lambda s: np.sort(s)[-10:].mean()}
fracs = (0.0005, 0.002, 0.01, 0.05, 0.2)

def aucs(seed, artefacts, shift, n_slides=200):
    rng = np.random.default_rng(seed)
    neg = [patch_scores(rng, 0.0, artefacts, shift) for _ in range(n_slides)]
    y = np.r_[np.zeros(n_slides), np.ones(n_slides)]
    res = {}
    for f in fracs:
        pos = [patch_scores(rng, f, artefacts, shift) for _ in range(n_slides)]
        res[f] = {name: roc_auc_score(y, [p(s) for s in neg + pos])
                  for name, p in pools.items()}
    return res

print("slide AUC, mean [min, max] over seeds 0-4")
for shift in (0.0, 0.5):                           # sd of log k between slides
    for artefacts in (0, 3):
        runs = [aucs(seed, artefacts, shift) for seed in range(5)]
        print(f"shift {shift}, {artefacts} artefact patches per slide on average")
        print("tumour patches" + "".join(f"{name:>22}" for name in pools))
        for f in fracs:
            cells = []
            for name in pools:
                v = np.array([r[f][name] for r in runs])
                cells.append(f"{v.mean():.3f} [{v.min():.3f}, {v.max():.3f}]")
            print(f"{f:6.2%} ({round(f * N_PATCHES):3d})" + "".join(f"{c:>22}" for c in cells))

slide AUC, mean [min, max] over seeds 0-4
shift 0.0, 0 artefact patches per slide on average
tumour patches                   max                  mean           top-10 mean
 0.05% (  1)  0.947 [0.930, 0.957]  0.552 [0.534, 0.575]  0.865 [0.843, 0.879]
 0.20% (  4)  1.000 [0.999, 1.000]  0.719 [0.701, 0.755]  1.000 [0.999, 1.000]
 1.00% ( 20)  1.000 [1.000, 1.000]  0.997 [0.996, 0.999]  1.000 [1.000, 1.000]
 5.00% (100)  1.000 [1.000, 1.000]  1.000 [1.000, 1.000]  1.000 [1.000, 1.000]
20.00% (400)  1.000 [1.000, 1.000]  1.000 [1.000, 1.000]  1.000 [1.000, 1.000]
shift 0.0, 3 artefact patches per slide on average
tumour patches                   max                  mean           top-10 mean
 0.05% (  1)  0.570 [0.547, 0.593]  0.553 [0.532, 0.569]  0.643 [0.600, 0.670]
 0.20% (  4)  0.693 [0.671, 0.707]  0.709 [0.673, 0.735]  0.908 [0.891, 0.925]
 1.00% ( 20)  0.865 [0.818, 0.882]  0.996 [0.994, 0.997]  1.000 [1.000, 1.000]
 5.00% (100)  0.963 [0.951, 0.976]  1.000 [1.000, 1.000]  1.000 [1.000, 1.000]
20.00% (400)  0.990 [0.982, 0.995]  1.000 [1.000, 1.000]  1.000 [1.000, 1.000]
shift 0.5, 0 artefact patches per slide on average
tumour patches                   max                  mean           top-10 mean
 0.05% (  1)  0.849 [0.837, 0.866]  0.494 [0.477, 0.525]  0.561 [0.541, 0.587]
 0.20% (  4)  0.970 [0.960, 0.978]  0.514 [0.493, 0.533]  0.775 [0.748, 0.797]
 1.00% ( 20)  0.995 [0.989, 0.999]  0.547 [0.507, 0.568]  0.994 [0.989, 0.999]
 5.00% (100)  0.998 [0.996, 1.000]  0.747 [0.706, 0.783]  1.000 [1.000, 1.000]
20.00% (400)  1.000 [0.999, 1.000]  0.987 [0.977, 0.994]  1.000 [1.000, 1.000]
shift 0.5, 3 artefact patches per slide on average
tumour patches                   max                  mean           top-10 mean
 0.05% (  1)  0.564 [0.525, 0.597]  0.517 [0.459, 0.573]  0.585 [0.559, 0.624]
 0.20% (  4)  0.673 [0.642, 0.701]  0.515 [0.457, 0.546]  0.778 [0.754, 0.812]
 1.00% ( 20)  0.871 [0.849, 0.880]  0.566 [0.549, 0.582]  0.992 [0.986, 0.998]
 5.00% (100)  0.966 [0.948, 0.981]  0.760 [0.737, 0.784]  0.999 [0.996, 1.000]
20.00% (400)  0.991 [0.979, 0.999]  0.985 [0.976, 0.991]  1.000 [1.000, 1.000]

With no shift and no artifacts, every rule works once tumor is more than a handful of tiles. Max pooling is the right operator for the smallest lesions: a single tumor tile out of 2,000 already separates most slides, which is the situation the MIL assumption describes. The mean needs about 20 tumor tiles, because an average over 2,000 tiles is very stable from slide to slide, so a 1% change in it is easy to detect.

The shift changes this for the mean alone. When the scores of normal tissue vary between slides, that variation is larger than the change a few tumor tiles make to the average, and the mean stays near chance at 1% tumor and needs about a fifth of the slide to recover; max pooling loses a little at the smallest fractions and the top-10 mean more, since it averages a single tumor tile with nine normal ones whose level now varies, but both recover by 1%, since they look only at the highest scores. The mean’s failure therefore comes from the assumed shift, whose size is ours. It agrees with Ilse et al.’s remark that the mean is a bad operator for tile scores, but their paper states this without such a model, and the table without the shift shows that the remark depends on how much negative slides differ from one another.

A few artifact tiles reverse the ranking at small tumor fractions. Max pooling now sees a high score on almost every negative slide, so its AUC falls to about 0.57 for a single tumor tile in both conditions and stays below the top-10 mean at every tumor fraction. The top-10 mean is robust as long as tumor tiles outnumber artifacts, and fails like the others when one tumor tile competes with three false alarms. Without the shift, the mean even overtakes the max from four tumor tiles, since a few artifacts barely move an average; with the shift, it remains the weakest rule. The general lesson, which is our reading of this synthetic setting rather than a result from the literature, is that the pooling rule encodes assumptions about how large the evidence is, how clean the classifier’s negatives are and how much negative slides differ; learned attention pooling, as in Ilse et al. and CLAM, lets the training data set that trade-off instead of fixing it.

Other tasks

Beyond diagnosis, slide-level models are trained to predict molecular properties and outcomes from H&E morphology. The foundation-model papers below evaluate, among other tasks, gene mutation prediction, molecular subtyping, organ transplant assessment and classification of up to 108 cancer types (Chen et al., 2024, arXiv version), and tumor mutation burden (Xu et al., 2024). Image retrieval, captioning and text-based search appear with vision-language models such as CONCH (Lu et al., 2024).

Pathology Foundation Models

From 2023, several groups trained general-purpose encoders on pathology images with self-supervised learning, so that one frozen encoder can supply tile features for many downstream tasks, in the sense of the foundation models of general vision. This is an emerging area; the facts below are as the papers report them, as of October 2026, and the papers differ in their evaluations, so their numbers are not directly comparable.

Model	Paper	Pretraining data (as reported)	Method
UNI	Chen et al., 2024	over 100 million images from over 100,000 diagnostic H&E slides, 20 major tissue types (Mass-100K)	ViT-Large trained with DINOv2
CONCH	Lu et al., 2024	over 1.17 million image–caption pairs, with histopathology images and biomedical text	CoCa-based image encoder, text encoder and multimodal decoder; contrastive and captioning objectives
Virchow	Vorontsov et al., 2024	1,488,550 H&E slides from 119,629 patients at one cancer center	ViT-H/14 (632 million parameters) trained with DINOv2
Prov-GigaPath	Xu et al., 2024	1.3 billion 256 × 256 tiles from 171,189 H&E and IHC slides, 28 cancer centers of one US health network	DINOv2 tile encoder, then a slide encoder pretrained as a masked autoencoder with LongNet

UNI and Virchow follow the DINO recipe of general vision: a student network matches an exponential-moving-average teacher on different crops of the same tile, with masked-patch prediction added in DINOv2 (Vorontsov et al., 2024, Methods). CONCH instead learns from paired images and captions, as CLIP does for web images, and so supports zero-shot classification and cross-modal retrieval (Lu et al., 2024, arXiv version). Prov-GigaPath adds a slide-level encoder over the whole sequence of tile embeddings, since a slide may contain tens of thousands of tiles and standard self-attention grows quadratically with their number (Xu et al., 2024).

The reported results are broad but come from each group’s own benchmark. UNI’s authors report outperforming earlier pathology encoders across their tasks, which number 33 in the arXiv version and 34 in the published abstract. CONCH’s evaluation suite counts 13 benchmarks in the arXiv version and 14 in the published abstract. Virchow’s authors report a pan-cancer detector reaching a specimen-level AUC of 0.95 across nine common and seven rare cancers, and Prov-GigaPath’s authors report state-of-the-art results on 25 of their 26 tasks. All four groups compare against earlier models on their own selection of tasks, and Virchow and Prov-GigaPath were pretrained on data from a single cancer center and a single health network respectively.

Datasets and Benchmarks

Most clinical slide collections are private, and public datasets are few and carefully described. The ones below are named with their official pages and licences; the facts come from their papers and pages, and no data were downloaded for this article.

Dataset	Content	Labels	Licence (as stated)
CAMELYON16	399 H&E sentinel lymph node slides from 2 Dutch centers: 270 training, 129 test	metastases outlined; slide labels confirmed with IHC where needed	CC0 (CAMELYON17 data page, which distributes both)
CAMELYON17	1,000 slides from 5 Dutch centers, organized as 200 patients of 5 slides	lesion outlines for 50 training slides; a pN-stage per patient	CC0 (CAMELYON17 data page)
PatchCamelyon	327,680 patches of 96 × 96 pixels at 10× from CAMELYON16 slides	binary: tumor in the central 32 × 32 region	CC0 (repository README)
PANDA	10,616 H&E prostate biopsies from 2 European centers (development set)	ISUP grade group per biopsy	CC BY-SA-NC 4.0, as the paper states it

CAMELYON16 (Ehteshami Bejnordi et al., 2017) posed two tasks: locating individual metastases, scored by the mean sensitivity at six false-positive rates from 1/4 to 8 per slide on the free-response ROC curve, and classifying each slide as containing metastases or not, scored by AUC. Later papers describe the release as 400 slides, with 130 test slides in Veeling et al. (2018) and 400 slides in total in Campanella et al. (2018, arXiv); the JAMA study analyzed 399 and 129. CAMELYON17 moved to patients: the task is a simplified pN-stage per patient, built from the size and number of metastases in five nodes and scored by five-class quadratic weighted kappa. The organizers note that they formed these artificial patients by mixing slides from several real patients, so the five slides of a patient should not be assumed to come from one person (CAMELYON17 evaluation page).

PANDA (Bulten et al., 2022) was a Gleason grading competition hosted on Kaggle in 2020, with 1,010 teams and 1,290 developers. Its design is unusual and instructive: the study was preregistered, the development set’s reference standard came from routine pathology reports (Dutch part) or one uropathologist (Swedish part), the internal validation set was graded by consensus of expert uropathologists, and 15 selected algorithms were reproduced by the organizers and run, unmodified apart from installation fixes, on US and European external validation sets the teams never saw. Agreement was measured as quadratically weighted kappa with the reference grade group.

Evaluation

What a pathology result means depends on four things the paper must state: the unit of evaluation (tile, lesion, slide or patient), the reference standard, the comparison, and the study design. The two benchmark studies above show why.

Units and metrics. Tile classification uses accuracy or AUC; detection uses free-response ROC analysis, which counts false positives per slide; slide and patient labels use AUC, sensitivity and specificity at an operating point, or Cohen’s kappa for ordinal grades. Nucleus segmentation uses instance-level measures such as panoptic quality, covered with cell segmentation. Pooling over tiles means that slide-level and tile-level numbers can differ greatly for the same model.

Reference standards. CAMELYON16’s slide labels were confirmed with IHC for difficult and apparently negative cases, which its authors describe as minimizing false-negative interpretations (Ehteshami Bejnordi et al., 2017). PANDA’s reference standards varied by set: routine reports or one uropathologist for training, consensus of three or four uropathologists for the internal validation set, a panel of six (two readers per case and a third for discordant cases) for the US external set, which also had IHC to aid tumor identification, and one uropathologist for the European external set (Bulten et al., 2022). A model can only be as good as the standard it is scored against, and a weaker standard in training (reports) than in testing (panels) is common.

Comparisons with readers, with their set-up. CAMELYON16’s slides were sampled retrospectively from 399 patients at the two hospitals. The best algorithm reached a slide-level AUC of 0.994 (95% CI 0.983–0.999). Eleven pathologists who read the corresponding glass slides with a microscope under a flexible two-hour limit, simulating routine workflow, had a mean AUC of 0.810 (range 0.738–0.884), and the difference was significant by multiple-reader multiple-case ROC analysis. One pathologist who reviewed the digital slides without a time limit, taking about 30 hours, had an AUC of 0.966, comparable to the top five algorithms (mean 0.960). The authors list the limits of this design: a test set enriched with metastases, and especially micrometastases, unlike routine case mix; one H&E slide per patient where practice examines several levels and orders IHC in equivocal cases; no rare co-occurring pathologies; and a simulation rather than clinical use. They state that validating such algorithms requires a prospective study, and that whether the approach has clinical utility will require evaluation in a clinical setting.

In PANDA, the selected algorithms agreed with the uropathologists’ reference with quadratically weighted kappas of 0.862 (95% CI 0.840–0.884) on the US external set and 0.868 (0.835–0.900) on the European external set. On 70 cases from the internal set, the algorithms agreed with the reference more than 13 general pathologists from 8 countries (kappa 0.876 against 0.765); on 237 US cases, they agreed about as well as 20 US general pathologists (0.828 against 0.820). On the external sets the representative algorithm’s specificity fell to 75.2% (US) and 84.3% (Europe), from 92.9% internally; the authors identify overdiagnosis of benign biopsies as the main error mode and attribute it in part to distribution shift and to thresholds chosen on the tuning set. All data were retrospective, and the authors call for prospective trials.

Statistics. Both studies report bootstrap confidence intervals, and CAMELYON16 compares algorithms with readers by multiple-reader multiple-case analysis, which treats readers and cases as random effects; the article on statistical significance in evaluation covers the general methods.

Regulation

In the United States, the two digital pathology device types below were classified through the FDA’s De Novo process, which, as both orders explain, can place a new kind of device in class I or II instead of the automatic class III; a device so classified can then serve as a predicate, so that later devices of the same type can use the 510(k) process. The milestones below are summarized only from the classification orders and the notice published in the Federal Register, as of October 2026; no legal conclusion is drawn from them.

• 20 April 2016. The FDA announced the availability of its guidance “Technical Performance Assessment of Digital Pathology Whole Slide Imaging Devices”, with recommendations on characterizing the technical performance of whole-slide imaging systems (Food and Drug Administration, 2016).
• 12 April 2017. On a De Novo request submitted by Philips Medical Systems Nederland B.V. on 1 December 2016 for the Philips IntelliSite Pathology Solution, the FDA classified the whole-slide imaging system into class II with special controls, codified as 21 CFR 864.3700 by a final order of 2 January 2018. The device type is identified as an automated digital slide creation, viewing and management system intended as an aid to the pathologist to review and interpret digital images of surgical pathology slides (Food and Drug Administration, 2018).
• 21 September 2021. On a De Novo request received from Paige.AI, Inc. on 31 December 2020 for Paige Prostate, the FDA classified the software algorithm device to assist users in digital pathology into class II, codified as 21 CFR 864.3750 by a final order of 2 February 2023. The device type evaluates scanned whole-slide images and provides information about the presence, location and characteristics of areas with clinical implications, to assist the user in determining a pathology diagnosis. Its special controls require, among other things, labeling that describes the device as an adjunct, training data that represent pre-analytical variables such as fixation and processing, multiple sites and demographics, an independent validation set, performance summaries stratified by confounders such as scanning equipment, and cybersecurity risk management (Food and Drug Administration, 2023).

These classifications concern the device types named in the orders and apply in the United States.

Challenges

• Scale. Gigapixel images force tiling, careful input/output and compressed storage; a whole-slide model must aggregate thousands of tiles, and Xu et al. (2024) note that earlier models often subsampled tiles and lost slide-level context.
• Weak and noisy labels. Slide labels from reports are cheap but coarse and imperfect. Several top PANDA teams cleaned labels automatically, excluding or relabeling slides whose grades disagreed strongly with the model (Bulten et al., 2022).
• Stain, scanner and site variation. Color and texture shift between laboratories and scanners (Tellez et al., 2019); the example above shows the two kinds of stain change that a fixed deconvolution matrix handles and does not handle.
• Subjective reference standards. Grading varies between pathologists, so results depend on whose grades are used, and panels of experts are expensive to assemble.
• Rare and unseen findings. A model trained to separate tumor from normal tissue may not recognize other pathologies; CAMELYON16’s authors name lymphoma, sarcoma and infection as examples not covered (Ehteshami Bejnordi et al., 2017).

Failure Modes and Risks

• Distribution shift. PANDA’s drop in specificity on external data, attributed by its authors partly to distribution shift between training and validation data, is a documented example (Bulten et al., 2022). Single-institution training data, as in several foundation models, carry the same risk.
• Artifacts. PANDA’s authors report that frequently miscalled cases often contained cutting artifacts and inflammatory and other biological processes, while noting that this analysis was not quantitative (Bulten et al., 2022). The pooling example shows how a few high-scoring artifact tiles can dominate a max-pooled slide score.
• Enriched test sets. Benchmarks that oversample positive and difficult cases, as CAMELYON16 did by design, measure discrimination but do not reproduce the predictive values of routine practice (Ehteshami Bejnordi et al., 2017).
• Population coverage. PANDA’s authors note that the countries in their study were predominantly white and that demographic data were not available for all datasets (Bulten et al., 2022).
• Over-reading heatmaps. Attention maps and tile heatmaps show where a model put weight, not a calibrated delineation; CLAM’s authors describe attention scores as relative within a slide (Lu et al., 2021, arXiv version).
• Benchmark artifacts. Constructed units of evaluation, such as CAMELYON17’s artificial patients, are fit for the challenge task but should not be read as real patient cohorts.

Deployment Considerations

• Formats and interoperability. Vendor formats differ, and reading them relies on libraries such as OpenSlide, whose support is unofficial. DICOM’s whole-slide objects allow slides to be stored and exchanged in picture archiving systems with other medical images, the goal Supplement 145 states (NEMA, 2010).
• Storage and compression. Even compressed, slides are large: Supplement 145’s example of 15 GB uncompressed falls to about 300 MB with JPEG 2000 (NEMA, 2010). Since lossy compression changes fine texture, it follows that compression settings are one more difference between training and deployment images.
• Computation. Feature-based pipelines such as CLAM reduce training cost by encoding tiles once (Lu et al., 2021, arXiv version); PANDA’s competition limited inference to 6 hours and 16 GB of GPU memory per submission (Bulten et al., 2022).
• Operating points. A threshold chosen on one dataset may not suit another. PANDA’s authors note that operating points can be adjusted to the intended use, for example favoring sensitivity for prescreening (Bulten et al., 2022).
• Image quality. Artifacts are a documented source of errors (see above), and the FDA’s special controls for analysis software ask manufacturers to describe situations, such as poor image quality, in which the device may fail or not reach its expected performance (Food and Drug Administration, 2023).
• Regulatory status and labeling. Whether a scanner or algorithm may be used clinically depends on its clearance in each jurisdiction; research models, including the foundation models above, carry no such status unless their makers obtain it.