Whole-Slide Image Classification

Whole-slide image classification predicts one label for a digitized microscope slide: metastasis present or absent in a lymph node section, the subtype of a lung or kidney cancer, the grade of a prostate biopsy, or, by combining several slides, a stage for a patient. The image is a whole-slide image of billions of pixels, far too large for a network’s input, while the label usually exists only for the slide as a whole, copied from the pathology report or assigned by a pathologist without outlining where the evidence lies. The task is therefore usually posed as weakly supervised learning, and most methods are forms of multiple-instance learning (MIL): the slide is cut into thousands of tiles (also called patches), each tile is encoded, and a learned pooling function turns the set of tiles into a slide prediction. Common names for the task include slide-level classification, weakly supervised WSI classification and MIL for histopathology.

Digital Pathology surveys the field around this task, including the standard MIL assumption, gated attention pooling, CLAM, PANDA’s results and an example that pools fixed tile scores by max, mean and top-k rules. Medical Image Classification treats what all medical classifiers share: the unit a label belongs to, report-mined labels, splits by patient and operating points. This article goes deeper into the slide-level problem itself: the MIL formulations and what each assumes, attention-based, critical-instance and transformer aggregators, where tile features come from, how reports become labels, and the protocols of CAMELYON16, CAMELYON17 and PANDA for evaluation at slide and patient level. Like its neighbours it describes methods, evidence and limits for practitioners and researchers, gives no diagnostic guidance, and attributes every reported result to its authors with the set-up behind it.

Problem Definition

A slide $X$ is a bag of tiles $\{x_1, \dots, x_K\}$, with $K$ varying from slide to slide, and one label $Y$. Each tile has a label too, $y_k$, but it is unknown during training. Ilse et al. (2018) state the MIL problem with two assumptions: the tiles have no ordering and no dependency among each other, and, for a binary label, the bag is negative if and only if all its instances are negative,

$$Y = \begin{cases} 0, & \text{if } \sum_k y_k = 0, \\ 1, & \text{otherwise}, \end{cases}$$

which is the standard MIL assumption written as $Y = \max_k y_k$ in Digital Pathology. Rather than optimizing that maximum directly, which they note gives vanishing gradients and suits only instance classifiers, Ilse et al. model the bag label as a Bernoulli variable with parameter $\theta(X) = P(Y = 1 \mid X)$ and train by maximizing the log-likelihood. Because the tiles are unordered, $\theta$ must be permutation-invariant. They use a form of the fundamental theorem of symmetric functions: a score $S(X)$ is symmetric if and only if it can be decomposed as

$$S(X) = g\Big(\sum_{x \in X} f(x)\Big),$$

and a related theorem gives arbitrarily close approximations with an element-wise maximum in place of the sum. Every MIL model is then three steps: a transformation $f$ of each tile, a permutation-invariant pooling $\sigma$, and a transformation $g$ of the pooled result.

The standard assumption fits detection: a single focus of metastasis makes a lymph node positive. Many slide labels do not follow it:

• Multi-class labels. Lu et al. (2021, arXiv version) point out that the assumption suits neither multi-class problems nor binary ones without an intrinsic positive and negative class, such as two cancer subtypes.
• Labels set by proportions. In prostate biopsies, specimens are categorized into one of five ISUP grade groups based on the distribution of Gleason growth patterns (Bulten et al., 2022), so a biopsy with some high-grade tissue need not belong to the highest group.
• Labels set by sizes and counts. CAMELYON17’s patient-level task assigns a simplified pN-stage from the number of positive lymph nodes and the sizes of the metastases in them (CAMELYON17 evaluation page), a rule over slides and over lesion sizes within them.
• Correlated tiles. Shao et al. (2021) argue that the independence assumption of earlier MIL methods is not entirely valid in many cases, since pathologists consider both the context around a region and the relations between regions, and propose a correlated MIL framework (see Approaches).

The formulation matters in practice, because the pooling function encodes it: the example below shows a max-pooled model that cannot express a proportion rule.

Input and Output

The input is the set of tissue tiles of a slide at one or more magnifications. Tissue detection, tiling and reading tiles at a target resolution are described in Whole-Slide Imaging. Bags are large and uneven. On CAMELYON16, Shao et al. (2021) obtained about 3.5 million 256 × 256 tiles at 20×, about 8,800 per slide on average, and Li et al. (2021) about 8,000 224 × 224 tiles per slide at 20× and 625 at 5×; Lu et al. (2021, arXiv version) report a mean of 41,802 tiles per slide at 40× for combined CAMELYON16 and CAMELYON17 slides, and Campanella et al. (2018, arXiv) about 1,000 tiles at 20× for a prostate needle biopsy.

The output is a slide-level probability or class distribution. Most methods also produce a score per tile, an instance score or an attention weight, which can be drawn over the slide as a heatmap. That map shows what the model weighted, not a delineation, as Digital Pathology notes for CLAM’s heatmaps; Ilse et al. (2018) present attention weights not as instance scores but as a proxy for them.

Variants

• Binary detection. Metastasis present or absent in a sentinel lymph node slide (CAMELYON16) or cancer present or absent in a needle biopsy (Campanella et al., 2018, arXiv).
• Subtyping. Two-class lung cancer subtyping (adenocarcinoma against squamous cell carcinoma) and three-class renal cell carcinoma subtyping on slides from The Cancer Genome Atlas (TCGA) are the usual public benchmarks in the papers below.
• Ordinal grading. The ISUP grade group of a prostate biopsy (PANDA), scored with quadratically weighted kappa, which weights each disagreement by the squared distance between the grades.
• Patient-level prediction. A label for a patient from several slides, as in CAMELYON17’s pN-stage over five lymph nodes.
• Molecular status and outcome. Mutation, biomarker or tumor mutation burden predicted from H&E slides with the same aggregators (Xu et al., 2024); Digital Pathology lists the foundation-model evaluations of this kind.

Challenges

• Share of evidence. The share of tiles that carry the evidence varies by task. Shao et al. (2021) and Li et al. (2021) both describe tumor as covering less than about 10% of the tissue in a positive CAMELYON16 slide, against more than 80% in the TCGA lung slides (and, in Shao et al., the kidney slides). Li et al. report that a classifier trained by giving every tile its slide’s label, which works on the TCGA lung data, did not converge on CAMELYON16 because of the many negative tiles in positive slides.
• Memory. End-to-end training of a tile encoder through a pooling of thousands of tiles does not fit in memory, so methods either freeze the encoder and train only the aggregator on stored features (CLAM, DSMIL, TransMIL) or train the encoder through a few selected tiles (Campanella et al., 2018, arXiv; PANDA’s leading teams sampled patches). Li et al. (2021) argue that both shortcuts can give sub-optimal tile features.
• Few labels per pixel. One bit per slide is a weak signal. Campanella et al. (2018, arXiv) trained their MIL model on nested training sets of 100 to 6,000 slides and found the validation error falling as the training set grew, concluding that MIL needs a large number of slides to generalize.
• Noisy labels. Report-derived and single-reader labels contain errors and subjective grades (see Weak Labels from Reports).
• Shift between sites. Staining and scanning differ between laboratories, which Stain Normalization treats, and a frozen encoder carries whatever sensitivity to color it learned.
• Evaluation units. Slides, specimens, patients and artificial patients are different units, and papers on the same dataset split it differently (see Evaluation Metrics).

Approaches

Instance-level and embedding-level MIL

Ilse et al. (2018) distinguish two places for the pooling. In the instance-level approach, $f$ is a tile classifier, its scores are pooled, and $g$ is the identity; the scores locate key tiles, but the tile classifier is trained without tile labels and adds its errors to the slide prediction. In the embedding-level approach, $f$ maps tiles to embeddings, the pooling gives a slide representation of fixed size, and a slide classifier $g$ scores it. Li et al. (2021) summarize the trade-off: the embedding-level approach is supervised directly by the bag label and usually more accurate, but makes key tiles harder to identify.

Fixed pooling operators include the maximum, the mean, the log-sum-exp (a smooth maximum), noisy-or and noisy-and, all differentiable (Ilse et al., 2018). The authors note that the maximum may suit instance scores but not embeddings, the converse of their remark on the mean that Digital Pathology quotes. Digital Pathology’s example shows how max, mean and top-k pooling of fixed scores depend on tumor size, artifacts and slide-to-slide shift.

Campanella et al.’s systems are instance-level. Their 2018 preprint (arXiv, titled “Terabyte-scale Deep Multiple Instance Learning for Classification and Localization in Pathology”, with three authors) trains a tile classifier under the standard assumption on the top-ranked tile of each slide, as Digital Pathology describes. On 12,160 prostate needle biopsies whose diagnoses came from the pathology reports, the best networks reached test AUCs of 0.976 (ResNet-34) and 0.977 (VGG11-BN) on 1,824 held-out slides, from a random 70/15/15 split. The 2019 Nature Medicine study with ten authors extended this to 44,732 slides from 15,187 patients and three cancer types (Campanella et al., 2019); later comparisons include its slide-level aggregation as MIL-RNN, which Shao et al. (2021) describe as based on a recurrent neural network.

Attention-based MIL

Ilse et al. (2018) replace the fixed pooling with a learned weighted average of embeddings $\mathbf{h}_k$,

$$\mathbf{z} = \sum_{k=1}^{K} a_k \mathbf{h}_k, \qquad a_k = \frac{\exp\{\mathbf{w}^\top \tanh(V \mathbf{h}_k)\}}{\sum_{j=1}^{K} \exp\{\mathbf{w}^\top \tanh(V \mathbf{h}_j)\}},$$

with parameters $\mathbf{w} \in \mathbb{R}^{L}$ and $V \in \mathbb{R}^{L \times M}$ (their Eq. 8). The weights sum to one, so the pooling does not depend on the bag size, and it satisfies the sum decomposition above with the weights absorbed into $f$. Because $\tanh$ is nearly linear on $[-1, 1]$, they also propose a gated version that multiplies $\tanh(V\mathbf{h}_k)$ element-wise by $\operatorname{sigm}(U\mathbf{h}_k)$ (Eq. 9, shown in Digital Pathology). Attention pooling sits between the two approaches: it pools embeddings, yet its weights point to the tiles that drove the prediction.

Their evaluation was small by later standards. On MNIST-bags (bags of digit images, positive if they contain a 9), attention pooling did much better than max and mean pooling when training bags were few, and all models converged to similar results with enough data. On two histopathology datasets of 58 and 100 images, cut into patches, the authors report that their approach outperformed the other methods; in their table for the breast cancer images, the embedding-level mean reached an AUC of 0.796, between plain attention (0.775) and gated attention (0.799), all within about two standard errors, while on the colon cancer images both attention variants reached 0.968 against 0.940 for the embedding-level mean.

CLAM (Lu et al., 2021) adds one attention branch per class and a clustering loss on the most and least attended tiles, as Digital Pathology describes. Its evaluation design is worth noting here: each of ten Monte Carlo folds split the cases 80/10/10 with all slides of a case in the same set, models were selected on the validation fold, and the ten models were also tested on independent cohorts scanned at the authors’ hospital, where averaging all ten rather than picking the best avoided what the authors call an illusion of data efficiency.

Critical-instance attention: DSMIL

Li et al. (2021) combine the two approaches in a dual-stream network (DSMIL). The first stream scores each tile embedding with a linear classifier and max-pools, which selects a critical instance $\mathbf{h}_m$. The second stream projects every embedding to a query $\mathbf{q}_i = W_q \mathbf{h}_i$ and an information vector $\mathbf{v}_i = W_v \mathbf{h}_i$, weights each tile by the softmax of its query’s inner product with the critical tile’s query,

$$U(\mathbf{h}_i, \mathbf{h}_m) = \frac{\exp \langle \mathbf{q}_i, \mathbf{q}_m \rangle}{\sum_k \exp \langle \mathbf{q}_k, \mathbf{q}_m \rangle}, \qquad \mathbf{b} = \sum_i U(\mathbf{h}_i, \mathbf{h}_m)\, \mathbf{v}_i ,$$

and scores the bag embedding $\mathbf{b}$; the final score averages the two streams. The authors relate this to self-attention, with the difference that only the critical tile is matched against the others and no separate key vectors are learned. Their motivation is that, when positive tiles are few, max pooling learns a decision boundary set by the highest-scoring tiles of each bag, while measuring distances to the critical instance delineates the positive tiles better. DSMIL also trains its tile encoder by self-supervised contrastive learning on the slides’ own tiles (below) and concatenates 5× and 20× features in a pyramid.

Transformer aggregation

Self-attention between all tiles models the correlations that MIL’s independence assumption leaves out, at a cost quadratic in the number of tiles. TransMIL (Shao et al., 2021) applies a transformer to the sequence of tile embeddings: the sequence is extended to a square number of tokens by repeating some embeddings and given a class token, then passed through two transformer layers whose self-attention is approximated by the Nyström method (linear in sequence length), and between them through a Pyramid Position Encoding Generator that reshapes the tokens into a 2-D grid and applies group convolutions with 3 × 3, 5 × 5 and 7 × 7 kernels as a conditional positional encoding; the class token gives the slide prediction. The authors justify the design with an approximation theorem for set functions and an entropy argument that correlated instances carry less uncertainty than independent ones. In their ablation on CAMELYON16 and TCGA lung data, removing position encoding lowered the AUC (0.842 against 0.931 on CAMELYON16), and sinusoidal encoding fell in between (0.894).

Whole-slide foundation models carry this idea further. Prov-GigaPath (Xu et al., 2024) pretrains a slide encoder over the full sequence of tile embeddings, using dilated self-attention (LongNet) to handle slides of tens of thousands of tiles, and feeds its contextualized embeddings to a shallow attention-MIL layer. In the authors’ ablation for cancer subtyping, replacing the slide encoder with attention-MIL pooling alone gave lower performance, which they take as evidence for modelling long-range dependencies across the slide. Digital Pathology describes the model and its pretraining data.

Tile features from pretrained encoders

Freezing the tile encoder reduces training to the aggregator, so the choice of encoder becomes a large part of the result. Three sources of features appear in the papers above:

• ImageNet-pretrained CNNs. CLAM encodes 256 × 256 tiles with a ResNet-50 pretrained on ImageNet, truncated after its third residual block and spatially averaged into 1,024 features; TransMIL uses the same features, reduced to 512 dimensions by a trainable layer.
• Self-supervised on the target slides. DSMIL trains a ResNet-18 with SimCLR, a contrastive method, on tiles from each dataset’s training slides.
• Pathology foundation models. Encoders pretrained by self-supervised learning on large pathology collections, such as UNI (Chen et al., 2024), which Digital Pathology describes with three others.

The comparisons show how much the features matter, under each paper’s set-up. With the DSMIL aggregator fixed, Li et al. (2021, Table 3) report a CAMELYON16 test AUC of 0.541 from ImageNet features, 0.715 from an encoder trained end to end with max pooling, and 0.894 from contrastive features (the same table lists 0.543 for features from the tile-label classifier that, in their text, did not converge on CAMELYON16); on TCGA lung, where positive slides are mostly tumor, the encoder trained by giving every tile its slide’s label did at least as well as the contrastive one (0.981 against 0.963). In UNI’s arXiv version (Extended Data Table 14), the same gated attention-MIL trained on CAMELYON16’s official training slides reached test AUROCs of 0.752 with ImageNet ResNet-50 features, 0.930 with CTransPath, 0.981 with REMEDIS and 0.975 with UNI (balanced accuracy 0.957 for UNI, the highest of the four). The authors excluded TCGA tasks without an external test cohort from their slide-level comparison, because CTransPath and REMEDIS were pretrained on almost all TCGA slides and could be inflated by that overlap; they describe such data contamination as a growing concern for foundation models trained on large public collections.

Weak labels from reports

Slide labels are cheap because they already exist. Campanella et al. (2018, arXiv) retrieved the diagnoses of their 12,160 biopsies from the original pathology reports in the hospital’s laboratory information system, and the 2019 study trained on reported diagnoses alone, avoiding pixel-level annotation, and evaluated on slides used without data curation (Campanella et al., 2019, abstract). The reference standard of PANDA’s Dutch training data was the routine pathology report, and of its Swedish training data one uropathologist (Bulten et al., 2022).

Such labels carry errors. When an expert pathologist reviewed the 55 false positives of Campanella et al.’s preprint, seven were considered highly suspicious for prostate cancer and six atypical, and in six of the 33 false negatives the pathologist found no sign of cancer, which suggests that some of the counted errors were errors of the report-derived labels. Several of PANDA’s top teams treated label noise from subjective grading as a key problem and cleaned labels automatically, excluding or relabeling training slides whose reference grade deviated strongly from the model’s prediction, typically more aggressively as training progressed (Bulten et al., 2022). What a model trained on such labels can claim therefore rests on a stronger reference standard at test time, such as PANDA’s expert panels for its validation sets or CAMELYON16’s immunohistochemistry-supported annotations (see below).

Datasets and Benchmarks

Digital Pathology tabulates the public datasets with their licences; this section gives the protocols that make results comparable, or not.

CAMELYON16

CAMELYON16 posed two tasks on H&E sentinel lymph node slides from two Dutch centers (Ehteshami Bejnordi et al., 2017). Slide classification (task 2) asks for a probability that each slide contains any metastasis, scored by ROC AUC; lesion detection (task 1) is scored by free-response ROC analysis. The 270 training slides (110 with metastases, 160 without) came with outlines of every metastasis, and the 129 test slides (49 with metastases, 80 without) were withheld; the reference standard used cytokeratin immunohistochemistry for difficult and apparently negative cases. Slides with only isolated tumor cells, whose clinical value the authors call disputed, were excluded. The organizers describe it as the first challenge using whole-slide images in histopathology (CAMELYON16 home page).

Because the challenge supplied lesion outlines, its submissions could train on annotated tumor regions; the top slide-level AUC of 0.994 (95% CI 0.983–0.999) came from an ensemble of two GoogLeNet networks with stain standardization, extensive augmentation and hard negative mining (Ehteshami Bejnordi et al., 2017, Table 2), and Digital Pathology gives its comparison with pathologists. Weakly supervised papers reuse the same test slides with slide labels only; Li et al. (2021) state that they ignore the pixel-level annotations in training. Their numbers therefore answer a different question from the challenge’s leaderboard. The papers also count the release differently: 270 training and 130 test slides in Shao et al. (2021), 271 and 129 in Li et al. (2021), and 399 slides after removing one test slide described as mislabeled in UNI’s arXiv version.

CAMELYON17

CAMELYON17 moved the unit to the patient (CAMELYON17 data and evaluation pages). Its 1,000 slides from five Dutch centers are organized as 100 training and 100 test patients of five slides, one lymph node per slide; lesion outlines were provided for 50 training slides and CAMELYON16’s slides were reused as lesion-level training data. Each patient receives a simplified pN-stage:

Stage	Rule (as the challenge states it)
pN0	no micro-metastases, macro-metastases or isolated tumor cells
pN0(i+)	only isolated tumor cells
pN1mi	micro-metastases, no macro-metastases
pN1	metastases in 1–3 nodes, at least one a macro-metastasis
pN2	metastases in 4–9 nodes, at least one a macro-metastasis

Macro-metastases are larger than 2.0 mm; micro-metastases are larger than 0.2 mm or more than 200 cells but smaller than 2.0 mm; isolated tumor cells are smaller and do not count as positive nodes. Submissions are ranked by five-class quadratic weighted kappa, with ties broken by kappa on the most severe stages first. A patient-level prediction therefore needs more than a slide score: lesion sizes per slide, then counting across slides, over artificial patients assembled from several real ones (Digital Pathology quotes the organizers’ warning). CLAM’s authors used only the 500 training slides, whose slide-level labels were public, and combined them with CAMELYON16 into a binary slide task (Lu et al., 2021, arXiv version).

PANDA

PANDA (Bulten et al., 2022) evaluated ISUP grade group prediction for prostate biopsies with a design that separated developers from evaluation. Teams trained on a development set of 10,616 biopsies from 2,113 patients and saw scores on a tuning set of 393 biopsies; final ranking used an internal validation set of 545 biopsies, blinded, with all samples of a patient assigned to either development or validation; 15 selected algorithms were then reproduced by the organizers and run on external sets of 741 (US) and 330 (EU) biopsies prepared and scanned by other laboratories. The metric was quadratic Cohen’s kappa against each set’s reference standard, with bootstrap confidence intervals resampling both algorithms and cases. Unlike the frozen-feature pipelines above, most leading teams trained end to end on a sample of patches per biopsy, as Digital Pathology describes with the results and the drop in specificity on external data.

Evaluation Metrics

Slide-level discrimination. Binary tasks report AUC; multi-class tasks report the mean or macro-averaged one-versus-rest AUC, as TransMIL and CLAM do for TCGA subtyping. Accuracy at a fixed threshold of 0.5, as in Shao et al. (2021), depends on calibration and class balance; UNI’s authors report balanced accuracy. Ordinal grades use quadratically weighted kappa, and patient stages in CAMELYON17 likewise. Localization, when outlines exist, is scored separately: Li et al. (2021) report CAMELYON16’s FROC score, the mean sensitivity at 1/4 to 8 false positives per slide, for attention maps of weakly supervised models (0.430 for DSMIL against 0.525 for a fully supervised tile classifier in their Table 1).

The unit and the split. Slides from one patient share tissue, staining and label, so a split by slide can leak; Medical Image Classification shows the size of this effect. Practice varies in the papers above: CLAM placed all slides of a case in one set, TransMIL ensured for its TCGA data that no patient’s slides fell in both training and test, Prov-GigaPath kept the largest slide of each patient for its mutation tasks, PANDA split by patient, and Campanella et al.’s preprint describes a random split of slides. Results computed per slide and per patient are different numbers, and a claim about patients needs a patient-level evaluation.

Comparability across papers. The same aggregator on the same test slides gives different numbers in different papers, because features, magnification, tiling and training differ:

Pooling on CAMELYON16 (test AUC)	Li et al. (2021), SimCLR features, Table 1	Shao et al. (2021), ImageNet ResNet-50 features, Table 1
Mean pooling	0.762	0.465
Max pooling	0.864	0.857
Attention MIL (Ilse et al.)	0.865	0.876
DSMIL	0.894	0.818
TransMIL	not reported	0.931

Each paper’s own method comes out best among the MIL methods in its own table. UNI’s ImageNet-feature attention-MIL (0.752 AUROC) and CLAM’s 0.953 on combined CAMELYON16 and CAMELYON17 slides with ten cross-validated folds add further set-ups, so a CAMELYON16 number means little without the features and split behind it. Test sets of 129 or 130 slides also give wide intervals: UNI’s balanced accuracy of 0.957, for example, has a 95% interval of 0.911–0.991. Statistical significance in evaluation covers the methods.

Example: Attention Pooling Against Fixed Pooling Rules

Digital Pathology’s example pools fixed tile scores. The script below trains the slide models from slide labels alone, on features from a frozen encoder, which is the pipeline of CLAM, DSMIL and TransMIL in miniature. Each synthetic slide has 100 tiles of 16 × 16 pixels showing hematoxylin optical density, the quantity color deconvolution recovers: normal tiles hold a few small nuclei, tumor tiles more and larger ones, high-grade tumor tiles more and larger still, and artifact tiles (ink or a fold) a dark band. Every slide has its own stain strength, a multiplicative factor with a log-normal spread of 0.25, and on average two artifact tiles. The frozen encoder is a bank of four Laplacian-of-Gaussian blob filters, whose mean and maximum responses and the tile’s mean and spread give ten features, standardized on the training tiles.

Two label rules stand for two formulations. In scenario A, the standard MIL assumption, a positive slide contains 1–5 tumor tiles and a negative none. In scenario B, a proportion rule in the spirit of grade groups, every slide is 20–60% tumor, and a slide is positive when 60–90% of its tumor tiles are high-grade and negative when 10–40% are. Five slide models share a trainable tile embedding: instance-level scores pooled by max, mean or the mean of the five highest (the rules of Digital Pathology’s example, now trained), the embedding-level mean, and gated attention (Ilse et al., 2018, Eq. 9). All train for 40 epochs with Adam on 240 slides and are tested on 240 others; the table gives the test slide AUC and, for models with tile scores, how well those scores rank tumor tiles within positive test slides. The generator and its parameters are ours, set before the first run; after that run, the only change was an added last column printing each seed’s slide AUC. It runs in under 30 seconds on a CPU with PyTorch 2.14.1, NumPy 2.5.4 and scikit-learn 1.9.1, with identical output on repeated runs.

import numpy as np
import torch
import torch.nn as nn
from sklearn.metrics import roc_auc_score

torch.set_num_threads(1)
T, P = 100, 16                       # tiles per slide, tile size in pixels
yy, xx = np.mgrid[0:P, 0:P]

def tiles(rng, kind, n):
    """n synthetic 16x16 hematoxylin-density tiles of one kind."""
    count, sigma, amp = {"normal": (3, 1.2, 0.6), "tumour": (6, 1.8, 0.9),
                         "high": (9, 2.4, 1.0), "artefact": (3, 1.2, 0.6)}[kind]
    img = 0.05 + rng.normal(0, 0.02, (n, P, P))
    for i in range(n):
        for _ in range(rng.poisson(count)):           # nuclei as Gaussian blobs
            cy, cx = rng.uniform(0, P, 2)
            img[i] += amp * np.exp(-((yy - cy)**2 + (xx - cx)**2) / (2 * sigma**2))
        if kind == "artefact":                        # ink or fold: a dark band
            r = rng.integers(0, P - 4)
            img[i, r:r + 4] += 1.2
    return img

def slide(rng, scenario, label):
    """One slide: (100, 16, 16) tiles and per-tile kind labels."""
    kinds = ["normal"] * T
    if scenario == "A":                               # standard MI assumption
        n_tum = rng.integers(1, 6) if label else 0    # 1-5 tumour tiles
        kinds[:n_tum] = ["tumour"] * n_tum
    else:                                             # share of high grade
        n_tum = int(rng.uniform(0.2, 0.6) * T)
        share = rng.uniform(0.6, 0.9) if label else rng.uniform(0.1, 0.4)
        n_high = int(round(share * n_tum))
        kinds[:n_tum] = ["high"] * n_high + ["tumour"] * (n_tum - n_high)
    n_art = min(rng.poisson(2), T - n_tum)
    kinds[T - n_art:] = ["artefact"] * n_art
    img = np.concatenate([tiles(rng, k, 1) for k in kinds])
    img *= np.exp(rng.normal(0, 0.25))                # slide-level stain strength
    return img, np.array([k in ("tumour", "high") for k in kinds])

def log_kernel(s, size=9):                            # negative LoG, scale-normalized
    r = np.arange(size) - size // 2
    r2 = r[:, None]**2 + r[None, :]**2
    k = (1 - r2 / (2 * s * s)) * np.exp(-r2 / (2 * s * s))
    return torch.tensor((k - k.mean()) / s**2, dtype=torch.float32)

ENC = torch.stack([log_kernel(s) for s in (1, 1.5, 2, 3)])[:, None]

def encode(img):                                      # frozen encoder: 10 features per tile
    x = torch.tensor(img, dtype=torch.float32).reshape(-1, 1, P, P)
    m = torch.relu(nn.functional.conv2d(x, ENC, padding=4))
    f = torch.cat([m.mean((2, 3)), m.amax((2, 3)), x.mean((2, 3)), x.std((2, 3))], 1)
    return f.reshape(-1, T, 10)

class MIL(nn.Module):
    def __init__(self, pool):
        super().__init__()
        self.pool = pool
        self.embed = nn.Sequential(nn.Linear(10, 32), nn.ReLU())
        self.head = nn.Linear(32, 1)
        self.V, self.U, self.w = nn.Linear(32, 16), nn.Linear(32, 16), nn.Linear(16, 1)

    def forward(self, x):                             # x: (slides, tiles, 10)
        h = self.embed(x)
        if self.pool == "attention":                  # gated attention, Ilse et al. eq. 9
            a = torch.softmax(self.w(torch.tanh(self.V(h)) * torch.sigmoid(self.U(h))), 1)
            return self.head((a * h).sum(1)).squeeze(-1), a.squeeze(-1)
        if self.pool == "emb-mean":
            return self.head(h.mean(1)).squeeze(-1), None
        s = self.head(h).squeeze(-1)                  # instance level: one score per tile
        if self.pool == "inst-max":
            return s.amax(1), s
        if self.pool == "inst-mean":
            return s.mean(1), s
        return s.topk(5, 1).values.mean(1), s         # inst-top5

def dataset(rng, scenario, n):
    data = [slide(rng, scenario, i % 2) for i in range(n)]
    return (encode(np.stack([d[0] for d in data])),
            torch.tensor([i % 2 for i in range(n)], dtype=torch.float32),
            np.stack([d[1] for d in data]))

POOLS = ["inst-max", "inst-mean", "inst-top5", "emb-mean", "attention"]

def run(seed, scenario):
    rng = np.random.default_rng(seed)
    torch.manual_seed(seed)
    xtr, ytr, _ = dataset(rng, scenario, 240)
    xte, yte, tumour = dataset(rng, scenario, 240)
    mu, sd = xtr.reshape(-1, 10).mean(0), xtr.reshape(-1, 10).std(0)
    xtr, xte = (xtr - mu) / sd, (xte - mu) / sd       # standardize with training tiles
    res = {}
    for pool in POOLS:
        model = MIL(pool)
        opt = torch.optim.Adam(model.parameters(), lr=1e-3)
        for epoch in range(40):
            for idx in torch.randperm(240).split(16):
                loss = nn.functional.binary_cross_entropy_with_logits(model(xtr[idx])[0], ytr[idx])
                opt.zero_grad(); loss.backward(); opt.step()
        with torch.no_grad():
            logit, tile = model(xte)
        res[pool, "slide"] = roc_auc_score(yte, logit)
        if tile is not None:                          # do tile scores find the tumour?
            pos = yte.numpy() == 1
            res[pool, "tile"] = np.mean([roc_auc_score(t, s) for t, s in
                                         zip(tumour[pos], tile[pos].numpy())])
    return res

for scenario in ("A", "B"):
    runs = [run(seed, scenario) for seed in range(5)]
    print(f"scenario {scenario}: mean [min, max] over seeds 0-4")
    print(f"{'pooling':10s} {'slide AUC':>22s} {'tumour-tile AUC':>24s}   slide AUC, seeds 0-4")
    for pool in POOLS:
        cells = []
        for key in ("slide", "tile"):
            v = np.array([r.get((pool, key), np.nan) for r in runs])
            cells.append("" if np.isnan(v).all() else
                         f"{v.mean():.3f} [{v.min():.3f}, {v.max():.3f}]")
        per_seed = " ".join(f"{r[pool, 'slide']:.2f}" for r in runs)
        print(f"{pool:10s} {cells[0]:>22s} {cells[1]:>24s}   {per_seed}")

scenario A: mean [min, max] over seeds 0-4
pooling                 slide AUC          tumour-tile AUC   slide AUC, seeds 0-4
inst-max     0.879 [0.501, 0.987]     0.782 [0.095, 0.989]   0.97 0.97 0.50 0.96 0.99
inst-mean    0.722 [0.599, 0.831]     0.956 [0.920, 0.977]   0.83 0.70 0.60 0.77 0.70
inst-top5    0.739 [0.454, 0.954]     0.614 [0.042, 0.992]   0.45 0.90 0.90 0.49 0.95
emb-mean     0.746 [0.657, 0.814]                            0.77 0.76 0.66 0.81 0.72
attention    0.883 [0.540, 0.985]     0.789 [0.021, 0.987]   0.97 0.96 0.95 0.54 0.99
scenario B: mean [min, max] over seeds 0-4
pooling                 slide AUC          tumour-tile AUC   slide AUC, seeds 0-4
inst-max     0.680 [0.480, 0.764]     0.561 [0.068, 0.854]   0.71 0.76 0.48 0.74 0.70
inst-mean    0.908 [0.829, 0.936]     0.884 [0.838, 0.983]   0.92 0.94 0.93 0.83 0.92
inst-top5    0.741 [0.514, 0.865]     0.674 [0.042, 0.968]   0.69 0.86 0.51 0.81 0.82
emb-mean     0.927 [0.898, 0.942]                            0.92 0.94 0.94 0.90 0.94
attention    0.963 [0.945, 0.976]     0.964 [0.957, 0.970]   0.98 0.97 0.96 0.94 0.96

Scenario A: the standard assumption. When the trained models work, max pooling and attention reach slide AUCs of 0.95–0.99 with only 1–5 tumor tiles in 100 and about two artifact tiles per slide on average. In Digital Pathology’s example, a few artifacts brought fixed max pooling down to an AUC of about 0.57 for a single tumor tile; here the tile scorer is trained on negative slides that contain artifacts, which is consistent with the learned max ignoring them, though the script does not measure artifact scores directly. But training is not reliable at this scale: max pooling and attention each fail in one seed of five (different seeds), with slide AUCs near 0.5, and their minimum tumor-tile AUCs of 0.10 and 0.02 show that in at least one seed the highest scores or weights went to tiles other than tumor; top-5 pooling fails in two seeds. The mean-based rules never fail but average only 0.72–0.75. The instance-mean model’s tile scorer is good, ranking tumor tiles first in every seed (tumor-tile AUC 0.92 or more), yet averaging 100 scores lets the slide-to-slide stain variation of the normal tiles outweigh a few tumor tiles: the effect Digital Pathology’s example shows for fixed scores, appearing again after training. A reasonable reading, which this run does not test, is that a mean passes gradient to every tile while max and attention learn mainly through a few tiles, so the latter need more slides to find the evidence reliably, in line with Campanella et al.’s dataset-size experiment and Ilse et al.’s remark that attention acts as a filter on gradient updates.

Scenario B: a proportion rule. Here max pooling cannot express the label: both classes contain high-grade tiles, and a maximum answers only whether one exists, so its AUC stays at 0.48–0.76 and top-5 pooling is not much better. The means do well (0.83–0.94), but tumor fraction varies from 20% to 60% between slides, so the average mixes the amount of tumor with its grade. Attention is the best and the most stable model (0.94–0.98), and its weights pick out tumor tiles of both grades (tumor-tile AUC 0.96); in our reading, a softmax-weighted average restricted to tumor tiles estimates the share of high grade among them, which is the quantity the label depends on. The two scenarios together illustrate that the formulation determines which pooling fits, and that attention fitted both rules, at the price, in this small setting with 120 positive training slides, of a failed run under the standard assumption. Real slides have thousands of tiles, learned encoders and labels that follow neither rule exactly; the example illustrates the mechanisms, not the size of any effect on real data.

Applications

Slide-level classifiers are developed for metastasis detection in lymph nodes, cancer detection and grading in biopsies, subtyping of resected tumors, and prediction of molecular status, outcomes and treatment response from routine H&E slides; Digital Pathology describes these uses, the foundation-model evaluations and the regulatory path of analysis software in the United States. Campanella et al. (2019) report in their abstract that applying their system would allow pathologists to exclude 65–75% of slides while retaining 100% sensitivity, a projection stated in the abstract (the only part read here), not a measured clinical effect. Patient-level labels such as CAMELYON17’s stages show the step from slides to the decisions they inform. How such systems are evaluated beyond retrospective test sets, in reader studies and prospective trials, is the subject of Clinical Validation and Regulation of Medical Imaging AI.

Open Problems

• Comparable evaluation. Papers re-split public datasets, count their slides differently and use different features, so the same aggregator scores 0.465 or 0.762 as mean pooling and 0.865 or 0.876 as attention MIL on CAMELYON16 test slides. Reporting the features, magnification, split unit and confidence intervals with every number is the minimum for comparison.
• Formulations beyond the standard assumption. Grades, stages and molecular labels depend on proportions, sizes, counts and spatial relations; aggregators that model these explicitly, and tests that tell which rule a model has learned, are open.
• Patient-level prediction. Most methods classify slides; combining slides, specimens and their reports into a patient-level prediction, without the artificial patients of CAMELYON17, needs datasets organized by patient.
• Label quality. Training on report-derived labels scales, but the label noise and the strength of the test reference standard limit what can be claimed (Bulten et al., 2022).
• Encoders and contamination. Frozen foundation-model features dominate current results, and pretraining on public collections such as TCGA overlaps the benchmarks built from them (Chen et al., 2024, arXiv version).
• Attention is not localization. Attention weights rank tiles within a slide; whether high weights correspond to the lesion has to be measured where outlines exist, as with CAMELYON16’s FROC score.