Vision Systems & Engineering

MONAI

The MONAI Consortium's open-source, PyTorch-based framework for deep learning on medical images, with its transforms and MetaTensor, datasets, networks, losses and metrics, sliding-window inference, workflows, bundles and Auto3DSeg, the defaults that change results in release 1.6.1, and a NumPy rebuild of its blended sliding-window inference.

intermediate

MONAI (Medical Open Network for AI) is an open-source framework for deep learning on medical images, built on PyTorch and released under the Apache License 2.0 by the MONAI Consortium. Its README describes it as part of the PyTorch Ecosystem and lists among its features flexible preprocessing for multi-dimensional medical data, compositional APIs that fit into existing workflows, domain-specific networks, losses and metrics, and multi-GPU, multi-node data parallelism. It is used to build training and inference pipelines for tasks such as medical image segmentation, where volumes carry a physical geometry and can be larger than a network’s input window.

This article describes MONAI 1.6.1, published on 27 September 2026, from its source code and the documentation sources in its repository at that tag (commit f2ec87a). It covers the module map, the defaults that change numbers, the versions and requirements, and an example that rebuilds MONAI’s blended sliding-window inference in NumPy and measures how window overlap and weighting change the stitched result. MONAI itself was not installed or run; the example tests our port.

Purpose

MONAI packages as PyTorch-compatible components the steps of a medical imaging pipeline: reading volumes together with their voxel-to-world geometry, reorienting and resampling them, normalizing intensities by modality, sampling patches with control over class balance, applying the same random transform to an image and its label, running a 3D network over a volume larger than its input window, inverting the preprocessing on the prediction, and scoring with metrics such as Dice and Hausdorff distance. Its modules document says the building blocks are decoupled so that they can be used inside existing PyTorch programs, while the workflow and bundle APIs assemble them into complete training and evaluation pipelines.

The README states three aims: a community of academic, industrial and clinical researchers working on a common foundation; end-to-end training workflows for healthcare imaging; and a standardized way to create and evaluate deep learning models. The README asks users to cite Cardoso et al. (2022), the framework paper, which was not read for this article.

Core Capabilities

The modules document (docs/source/modules.md at 1.6.1) groups MONAI into the areas below.

Transforms, MetaTensor and IO

Transforms come in two styles: array transforms, which take one array, and dictionary transforms (with a d suffix, such as Spacingd), which apply the same operation to several keys, such as an image and its segmentation. Patch sampling includes weighted and class-balanced sampling from user-specified weight maps. Image IO has several built-in backends; the installation guide names NiBabel for NIfTI files, and the package’s optional dependencies include itk, nibabel, pydicom, openslide-python and tifffile. The DICOM and NIfTI formats themselves are described in DICOM, and their geometry and the LPS and RAS conventions in Medical Image Preprocessing.

MetaTensor is MONAI’s core data structure: a PyTorch tensor that carries metadata, including the affine that maps voxel indices to world coordinates, so that pipelines can use it. Transforms record the operations they apply, which lets the mini-batch output of a model be decollated into per-item tensors and inverted back to an earlier stage of preprocessing; the modules document links a tutorial on inverse transforms and test-time augmentation. Lazy resampling, introduced in MONAI 1.2 and still described as experimental in the 1.6.1 documentation, turns transforms that can be written as homogeneous matrices into pending operations, which are fused and carried out together when they are needed, which the documentation says can reduce execution time, memory and the noise and artifacts of repeated resampling. Determinism is controlled through the Randomizable API and set_determinism.

Datasets and data loading

MONAI extends PyTorch’s Dataset and DataLoader. Multi-threaded cache datasets keep loaded and preprocessed data across epochs; the cache can be persistent or dynamic (SmartCacheDataset) and reused across experiments. ThreadDataLoader runs transforms in a separate thread; the modules document offers it for light transforms on data cached in RAM, where the worker processes of PyTorch’s DataLoader may add unnecessary inter-process communication time. Dataset classes such as MedNISTDataset, DecathlonDataset and TciaDataset include downloading; PatchDataset, ZipDataset, CSVDataset and partition_dataset cover patch-level processing, multiple sources and cross-validation splits.

Networks, losses and metrics

Network layers and blocks support 1D, 2D and 3D inputs. The applications document describes reimplementations such as Swin UNETR, DiNTS, DeepGrow, DeepEdit, NuClick, RetinaNet-based 2D and 3D detection and learning-based registration; release 1.6.1 added NaViT and Hyena-based blocks. Some routines have C++/CUDA extensions (a resampler, a conditional random field, permutohedral bilateral filtering and a Gaussian mixture module). Losses include DiceLoss, GeneralizedDiceLoss, TverskyLoss and DiceFocalLoss, with AUCMarginLoss and MCCLoss added in 1.6.0. Metrics include mean Dice, ROC AUC, confusion matrices, Hausdorff and surface distances; MONAI’s Hausdorff conventions are covered in Hausdorff Distance, and the soft Dice loss in Dice Coefficient.

Inference

The monai.inferers module provides SimpleInferer, SlidingWindowInferer, SlidingWindowInfererAdapt, PatchInferer, SliceInferer, SaliencyInferer and inferers for diffusion models, plus splitters and mergers for patch-based inference. Sliding-window inference runs a network over overlapping windows of a large volume and blends the outputs; the modules document presents it as a way to keep memory requirements flexible. Its placement and blending rules are spelled out below, because they are what the example rebuilds.

Workflows, bundles, federated learning and Auto3DSeg

monai.engines provides trainers and evaluators compatible with PyTorch Ignite’s Engine and event handlers, including an EnsembleEvaluator; the modules document notes that most other modules can be used without Ignite, for example with PyTorch Lightning or MLflow. A bundle packages one model: per the modules document, the weights of a single network as a pickled state dictionary, optionally a TorchScript and/or an ONNX object, and JSON files with metadata, the configuration for training, inference and post-processing, a description and legal information. The MONAI Model Zoo distributes models in this format. For federated learning, MonaiAlgo implements the abstract ClientAlgo class to run bundles on a federated platform; the modules document states that NVIDIA FLARE has built an integration with ClientAlgo. Auto3DSeg analyses a dataset’s intensities, dimensionality and resolution, generates algorithms as bundles from templates, trains them, and selects algorithms for an ensemble by ranking the trained checkpoints. The nnU-Net article reports a benchmark by the nnU-Net developers in which three Auto3DSeg models did not reach the original nnU-Net on any of six datasets.

Conventions and Defaults in 1.6.1

These are read from the 1.6.1 source. Several differ from what a user of another framework might assume.

Sliding-window inference. sliding_window_inference(inputs, roi_size, sw_batch_size, predictor, overlap=0.25, mode="constant", sigma_scale=0.125, padding_mode="constant", cval=0.0, ...) expects channel-first input with a batch dimension, (N,C,…)(N, C, \ldots); SlidingWindowInferer has the same defaults. With rdr_d the window size along axis dd and oo the overlap, the step is

id=int⁡(rd(1−o)),i_d = \operatorname{int}\big(r_d (1 - o)\big),

truncated, set to 1 if it comes out as 0, and set to rdr_d when the window spans the whole axis (_get_scan_interval). Windows start at k idk\,i_d, for k=0,1,…k = 0, 1, \ldots until a window reaches the far border, and a window that would overrun the border is shifted back to end on it (dense_patch_slices), so the last step along an axis can be shorter than the others. Each window’s prediction is multiplied by an importance map ww and added into an output volume, the maps are summed into a count map, and the output is divided by the count map:

y^(v)=∑k∋vwk(v) pk(v)∑k∋vwk(v).\hat{y}(v) = \frac{\sum_{k \ni v} w_k(v)\, p_k(v)}{\sum_{k \ni v} w_k(v)} .

For mode="constant" the map is all ones, so the output is the plain average of the windows covering a voxel. For mode="gaussian", compute_importance_map builds along each axis g(x)=exp⁡(−x2/2σ2)g(x) = \exp(-x^2 / 2\sigma^2) on x=−r−12,…,r−12x = -\tfrac{r-1}{2}, \ldots, \tfrac{r-1}{2} with σ=s r\sigma = s\,r, where ss is sigma_scale, in voxels of the window (not millimetres), takes the outer product over axes, and clamps the map from below at max⁡(min⁡w,10−3)\max(\min w, 10^{-3}). The map is built in float32 and cast to the dtype of the input. The input is padded, with cval 0 by default, only when the volume is smaller than the window; in raw CT intensities 0 is the value of water, not air, which is our reading of what that padding means for unnormalized CT (Hounsfield Units).

Two consequences follow from the formula (our derivation). At the default sigma_scale the edge weight along one axis is exp⁡ ⁣(−8(r−1)2/r2)\exp\!\big(-8 (r-1)^2 / r^2\big), about 5.5×10−45.5 \times 10^{-4} for r=32r = 32, so the clamp is active and every weight below 10−310^{-3} is raised to it: in 3D, every voxel of the window with ∑dxd2>2σ2ln⁡1000\sum_d x_d^2 > 2\sigma^2 \ln 1000. And for an even window no sample falls at x=0x = 0, so the map never reaches 1; the division by the count map makes that harmless. MONAI’s defaults differ from nnU-Net’s, which the nnU-Net article describes as half-patch overlap with a Gaussian weight of standard deviation one eighth of the patch size.

DiceLoss. DiceLoss defaults to include_background=True, sigmoid=False, softmax=False, squared_pred=False, jaccard=False, batch=False, reduction="mean" and smooth_nr = smooth_dr = 1e-5. For hard labels, compute_tp_fp_fn gives tp=∑p ttp = \sum p\,t, fp=∑p−tpfp = \sum p - tp and fn=∑t−tpfn = \sum t - tp, and the loss is

1−2 tp+snr2 tp+fp+fn+sdr,1 - \frac{2\,tp + s_{nr}}{2\,tp + fp + fn + s_{dr}},

computed per sample and per channel unless batch=True, which sums over the batch before dividing; reduction="mean" then averages over batch and channels. No activation is applied by default, so a user who passes logits must set sigmoid or softmax (or give other_act), as the docstring states; the background channel counts unless include_background=False.

Spacing. Spacing(pixdim, mode="bilinear", padding_mode="border", align_corners=False, dtype=np.float64) resamples in float64 by default and returns float32; points outside the input take the border value. With diagonal=True the output affine is reset to a diagonal matrix, which the docstring describes as the world system (RAS+ in NiBabel).

Orientation. Orientation(axcodes=...) has labels=None as its default in 1.6.1, under a deprecation decorator that, with neither a deprecation nor a replacement version given, treats the change as made and issues a FutureWarning whenever labels is not passed (deprecate_utils.py). When the input is a MetaTensor whose "space" metadata is LPS, the code uses the labels (("R", "L"), ("A", "P"), ("I", "S")); otherwise it passes labels=None on to NiBabel’s axcodes2ornt, which the docstring gives as (("L", "R"), ("P", "A"), ("I", "S")). A plain tensor’s affine is taken to be the identity, with a warning. The naming trap of three-letter orientation codes is explained in Medical Image Preprocessing.

ScaleIntensityRange. ScaleIntensityRange(a_min, a_max, b_min=None, b_max=None, clip=False, dtype=np.float32) maps [amin⁡,amax⁡][a_{\min}, a_{\max}] linearly to [bmin⁡,bmax⁡][b_{\min}, b_{\max}] (to [0,1][0, 1] when either of them is None, the default) and does not clip unless asked, so values outside a CT window are extrapolated rather than saturated by default; the window itself is described in Hounsfield Units.

Minimal Example

MONAI is not installed on the machine this article was written on. The example therefore ports the part of sliding_window_inference that places windows and blends their outputs to NumPy 2.3.5 and SciPy 1.16.3 and runs it on a synthetic volume. The outcomes test our port, not MONAI. The port follows _get_scan_interval (monai/inferers/utils.py), dense_patch_slices and compute_importance_map (monai/data/utils.py) at tag 1.6.1, and the accumulation and division of sliding_window_inference, for one code path only: one channel-first volume, batch size 1, sw_batch_size 1, a volume larger than the window (so no padding), no buffering, a single output, no process_fn and no precomputed roi_weight_map. Everything is float32, as in MONAI for a float32 input: the importance map, the output and the count map.

Data. A synthetic CT-like volume of 64×128×12864 \times 128 \times 128 voxels (NumPy order z, y, x; spacing nominally 1 mm and not used): air at −1000 HU, a body ellipsoid at 40 HU, an “organ” ellipsoid and three spheres of radius 7, 6 and 5 voxels at 120 HU, and Gaussian noise of 20 HU everywhere. No patient data or downloaded images are used. A data check, D1, compares each structure’s voxel count with 43πabc\tfrac{4}{3}\pi abc (bar 5 %), its mean intensity with the planned value (bar 4 standard errors), the noise standard deviation in the air with 20 (4 standard errors), and confirms that the inner structures neither touch each other nor leave the body.

Predictor. No network is trained or loaded. The predictor is a fixed filter, our analogue of a network whose convolutions pad with zeros: a Gaussian blur of σ=1.5\sigma = 1.5 voxels computed with mode="constant", cval=0 inside each window (SciPy’s default truncation gives a radius of 6 voxels), followed by a sigmoid, p=1/(1+e−(G−80)/10)p = 1/(1 + e^{-(G - 80)/10}), which is close to 1 in the 120 HU structures and close to 0 elsewhere. The reference is the same predictor applied to the whole volume. Inside a window the blur sees zeros beyond the window’s faces, as a zero-padded network would; on the volume’s outer faces the reference pads with zeros too, so only internal window faces can make the stitched output differ from the reference.

Conditions and measures. The window is 32332^3; overlap is 0, 0.25 (MONAI’s default), 0.5 and 0.75, each with constant and gaussian blending (sigma_scale 0.125). For each condition the script prints the number of windows (M1) next to the closed form ∏d(⌈(Sd−rd)/id⌉+1)\prod_d \big(\lceil (S_d - r_d)/i_d \rceil + 1\big), the RMS and maximum absolute error against the reference over all voxels (M2), and the Dice coefficient of the output thresholded at 0.5 against the thresholded reference (M3). It also prints the minimum of the Gaussian importance map (M4), and two checks: C1, that the count map has no zero and no window leaves the volume; C2, that with constant blending at overlap 0 the error is exactly 0 on every voxel whose ±6-voxel filter support stays inside its window.

Hypotheses, written before any code ran:

  • H1. At overlap 0 the two blending modes agree to float rounding, within 2−23≈1.19×10−72^{-23} \approx 1.19 \times 10^{-7}.
  • H2. The window count equals the closed form, exactly.
  • H3. With Gaussian blending, the error falls strictly as overlap rises, for the RMS and, separately, for the maximum.
  • H4. At every overlap above 0, Gaussian blending gives a smaller RMS and a smaller maximum error than constant blending.
  • H5. The minimum of the Gaussian map equals the clamp, float32(10−310^{-3}).

H1 and C2 hold by construction if the port is right. With sizes divisible by the window and no overlap, each voxel is covered by exactly one window, so constant blending returns that window’s prediction pp and Gaussian blending returns fl(fl(p w)/w)\mathrm{fl}(\mathrm{fl}(p\,w)/w), which differs from pp by at most p(2u+u2)p(2u + u^2) with u=2−24u = 2^{-24}; and a voxel far enough from its window’s faces sees the same inputs through the same filter passes as in the reference. Neither says anything about the merits of Gaussian blending; they check the port. H3 and H4 are the comparisons.

All bars were fixed before a planning script computed the values a correct port reaches on the pilot’s data. The pilot ran with noise seed 1; the first run, shown here, uses seed 0 and is otherwise the same script. The script runs in about one to two seconds and printed identical output on two runs. The code is the script as first run.

import os
os.environ["OMP_NUM_THREADS"] = "1"; os.environ["OPENBLAS_NUM_THREADS"] = "1"
import math
import numpy as np
from scipy.ndimage import gaussian_filter

SEED = 0
SHAPE, ROI = (64, 128, 128), (32, 32, 32)

# ---- MONAI 1.6.1 window placement and importance map, ported to NumPy -------------------
# Copyright (c) MONAI Consortium. Licensed under the Apache License, Version 2.0:
# https://github.com/Project-MONAI/MONAI/blob/f2ec87ae9b536baaeca1d6bcc2b6349369e2af8f/LICENSE
# Modified (ported): _get_scan_interval (monai/inferers/utils.py), dense_patch_slices and
# compute_importance_map (monai/data/utils.py) at tag 1.6.1, rewritten for NumPy arrays.
def get_scan_interval(image_size, roi_size, overlap):
    out = []
    for s, r in zip(image_size, roi_size):
        if r == s:
            out.append(int(r))
        else:
            i = int(r * (1 - overlap))
            out.append(i if i > 0 else 1)
    return tuple(out)

def dense_patch_slices(image_size, patch_size, scan_interval):
    starts = []
    for s, p, i in zip(image_size, patch_size, scan_interval):
        num = int(math.ceil(float(s) / i))
        n = next((d for d in range(num) if d * i + p >= s), None)
        n = n + 1 if n is not None else 1
        starts.append([k * i - max(k * i + p - s, 0) for k in range(n)])
    grid = np.asarray([g.flatten() for g in np.meshgrid(*starts, indexing="ij")]).T
    return [tuple(slice(int(a), int(a) + p) for a, p in zip(x, patch_size)) for x in grid]

def compute_importance_map(patch_size, mode, sigma_scale=0.125, dtype=np.float32):
    if mode == "constant":
        imp = np.ones(patch_size, np.float32)
    else:
        for k, p in enumerate(patch_size):
            x = np.arange(-(p - 1) / 2.0, (p - 1) / 2.0 + 1, dtype=np.float32)
            x = np.exp(x ** 2 / (-2 * (p * sigma_scale) ** 2))            # 1D Gaussian, float32
            imp = imp[..., None] * x[(None,) * k] if k > 0 else x
    min_non_zero = max(float(imp.min()), 1e-3)
    return np.maximum(imp.astype(np.float32), np.float32(min_non_zero)).astype(dtype)
# ---- end of ported code ------------------------------------------------------------------

def sliding_window(vol, predictor, overlap, mode):
    """Sum of window prediction x weight over windows, divided by the summed weights (float32)."""
    slices = dense_patch_slices(vol.shape, ROI, get_scan_interval(vol.shape, ROI, overlap))
    w = compute_importance_map(ROI, mode, dtype=vol.dtype)
    out, cnt = np.zeros(vol.shape, vol.dtype), np.zeros(vol.shape, vol.dtype)
    for sl in slices:
        cnt[sl] += w
    for sl in slices:
        out[sl] += predictor(vol[sl]) * w
    return out / cnt, slices, cnt, w

# ---- synthetic CT-like volume (HU), every parameter known --------------------------------
def ellipsoid(c, a):
    z, y, x = np.indices(SHAPE)
    return ((z - c[0]) / a[0]) ** 2 + ((y - c[1]) / a[1]) ** 2 + ((x - c[2]) / a[2]) ** 2 <= 1

STRUCT = [("body", (32, 64, 64), (28, 54, 58), 40.0), ("organ", (30, 58, 50), (14, 26, 22), 120.0),
          ("sphere1", (20, 40, 92), (7, 7, 7), 120.0), ("sphere2", (44, 88, 86), (6, 6, 6), 120.0),
          ("sphere3", (32, 100, 50), (5, 5, 5), 120.0)]
masks = {n: ellipsoid(c, a) for n, c, a, _ in STRUCT}
vol = np.full(SHAPE, -1000.0)                                         # air
for n, c, a, v in STRUCT:
    vol[masks[n]] = v
vol = (vol + np.random.default_rng(SEED).normal(0.0, 20.0, SHAPE)).astype(np.float32)

inner = ["organ", "sphere1", "sphere2", "sphere3"]
print(f"D1 data check (seed {SEED}):")
for n, c, a, v in STRUCT:
    region = masks[n] & ~np.any([masks[m] for m in inner], axis=0) if n == "body" else masks[n]
    count, V = int(masks[n].sum()), 4 / 3 * math.pi * a[0] * a[1] * a[2]
    z = abs(float(vol[region].mean()) - v) / (20 / math.sqrt(region.sum()))
    print(f"  {n:8s} voxels {count:7d} vs 4/3 pi abc {V:9.1f}: rel. diff {abs(count - V) / V:.4f} (bar 0.05); "
          f"mean {vol[region].mean():8.3f} HU vs {v:6.1f}: {z:.3f} SE (bar 4)")
air = ~masks["body"]
sd = float(vol[air].std())
overl = sum(int((masks[p] & masks[q]).sum()) for i, p in enumerate(inner) for q in inner[i + 1:])
n_out = sum(int((masks[m] & air).sum()) for m in inner)
print(f"  air noise std {sd:.4f} vs 20: {abs(sd - 20) / (20 / math.sqrt(2 * air.sum())):.3f} SE (bar 4); "
      f"overlapping structure voxels {overl}; structure voxels outside the body {n_out}")

# ---- predictor: our analogue of a zero-padded network (no training) ----------------------
def predictor(v):
    g = gaussian_filter(v, 1.5, mode="constant", cval=0.0)             # zeros beyond the window
    with np.errstate(over="ignore"):
        return (1.0 / (1.0 + np.exp(-(g - np.float32(80.0)) / np.float32(10.0)))).astype(np.float32)

ref = predictor(vol)                                                 # the whole volume at once
ref_mask = ref > 0.5

def dice(a, b):
    return 2 * np.count_nonzero(a & b) / (np.count_nonzero(a) + np.count_nonzero(b))

# C2 region: voxels whose +-6 voxel filter support stays inside their overlap-0 window
far = np.ones(SHAPE, bool)
for ax, n in enumerate(SHAPE):
    k = np.arange(n) % ROI[ax]
    ok = (k >= 6) & (k <= ROI[ax] - 7)
    ok |= (np.arange(n) < 6) | (np.arange(n) > n - 7)                  # outer faces pad like the reference
    far &= ok.reshape([n if d == ax else 1 for d in range(3)])

print("\n overlap  mode      windows  closed form   min count   outside       RMS error     max error      Dice")
res = {}
for ov in (0.0, 0.25, 0.5, 0.75):
    i = [int(r * (1 - ov)) for r in ROI]
    closed = math.prod(math.ceil((s - r) / ii) + 1 for s, r, ii in zip(SHAPE, ROI, i))
    for mode in ("constant", "gaussian"):
        out, slices, cnt, w = sliding_window(vol, predictor, ov, mode)
        outside = sum(any(s.start < 0 or s.stop > n for s, n in zip(sl, SHAPE)) for sl in slices)
        err = np.abs(out.astype(np.float64) - ref)
        res[ov, mode] = (out, math.sqrt(np.mean(err ** 2)), float(err.max()), w)
        print(f"  {ov:4.2f}   {mode:9s} {len(slices):6d} {closed:9d}   {cnt.min():.6e} {outside:6d}   "
              f"{res[ov, mode][1]:.6e}  {res[ov, mode][2]:.6e}  {dice(out > 0.5, ref_mask):.6f}")
        if ov == 0.0 and mode == "constant":
            c2 = float(err[far].max())

print(f"\nC2: constant, overlap 0, max error where the filter stays inside its window "
      f"({int(far.sum())} voxels): {c2:.3e} (bar 0)")
h1 = float(np.abs(res[0.0, "gaussian"][0] - res[0.0, "constant"][0]).max())
print(f"H1: overlap 0, max |gaussian - constant| = {h1:.3e} (bar 2^-23 = {2.0 ** -23:.3e})")
for k, name in ((1, "RMS"), (2, "max")):
    g = [res[o, "gaussian"][k] for o in (0.0, 0.25, 0.5, 0.75)]
    print(f"H3-{name}: gaussian error strictly falls with overlap: {all(a > b for a, b in zip(g, g[1:]))}")
    print(f"H4-{name}: gaussian < constant at 0.25, 0.5, 0.75: "
          f"{[res[o, 'gaussian'][k] < res[o, 'constant'][k] for o in (0.25, 0.5, 0.75)]}")
m4 = res[0.25, "gaussian"][3].min()
print(f"H5: minimum of the Gaussian map {m4:.10e}, float32(1e-3) {np.float32(1e-3):.10e}, "
      f"equal: {m4 == np.float32(1e-3)}; map dtype {res[0.25, 'gaussian'][3].dtype}")
D1 data check (seed 0):
  body     voxels  367197 vs 4/3 pi abc  367340.1: rel. diff 0.0004 (bar 0.05); mean   40.087 HU vs   40.0: 2.492 SE (bar 4)
  organ    voxels   33445 vs 4/3 pi abc   33543.8: rel. diff 0.0029 (bar 0.05); mean  120.065 HU vs  120.0: 0.594 SE (bar 4)
  sphere1  voxels    1419 vs 4/3 pi abc    1436.8: rel. diff 0.0124 (bar 0.05); mean  119.717 HU vs  120.0: 0.532 SE (bar 4)
  sphere2  voxels     925 vs 4/3 pi abc     904.8: rel. diff 0.0223 (bar 0.05); mean  120.074 HU vs  120.0: 0.112 SE (bar 4)
  sphere3  voxels     515 vs 4/3 pi abc     523.6: rel. diff 0.0164 (bar 0.05); mean  121.065 HU vs  120.0: 1.208 SE (bar 4)
  air noise std 20.0231 vs 20: 1.348 SE (bar 4); overlapping structure voxels 0; structure voxels outside the body 0

 overlap  mode      windows  closed form   min count   outside       RMS error     max error      Dice
  0.00   constant      32        32   1.000000e+00      0   5.900606e-02  9.780743e-01  0.871407
  0.00   gaussian      32        32   1.000000e-03      0   5.900606e-02  9.780743e-01  0.871407
  0.25   constant      75        75   1.000000e+00      0   3.561452e-02  7.163874e-01  0.961010
  0.25   gaussian      75        75   1.000000e-03      0   2.275047e-02  6.981412e-01  0.985336
  0.50   constant     147       147   1.000000e+00      0   3.985955e-02  7.294566e-01  0.945393
  0.50   gaussian     147       147   1.000000e-03      0   1.359214e-03  3.885257e-02  0.999546
  0.75   constant     845       845   1.000000e+00      0   3.090851e-02  4.564553e-01  0.970326
  0.75   gaussian     845       845   1.000000e-03      0   9.339062e-04  1.403332e-02  0.999461

C2: constant, overlap 0, max error where the filter stays inside its window (440128 voxels): 0.000e+00 (bar 0)
H1: overlap 0, max |gaussian - constant| = 5.960e-08 (bar 2^-23 = 1.192e-07)
H3-RMS: gaussian error strictly falls with overlap: True
H4-RMS: gaussian < constant at 0.25, 0.5, 0.75: [True, True, True]
H3-max: gaussian error strictly falls with overlap: True
H4-max: gaussian < constant at 0.25, 0.5, 0.75: [True, True, True]
H5: minimum of the Gaussian map 1.0000000475e-03, float32(1e-3) 1.0000000475e-03, equal: True; map dtype float32

Checks. The data check D1 held on every line: the largest relative count difference is 0.0223 (sphere2), the largest mean deviation 2.492 standard errors (body), the noise deviation 1.348 standard errors, and the structures are disjoint and inside the body. C1 held in all eight conditions: the count map’s minimum is 1 for constant blending and the clamp value for Gaussian blending, reached at the volume’s corners, which only one window covers. C2 held: on the 440,128 voxels whose filter stays inside their window, the stitched output equals the reference exactly, so all error comes from the bands next to internal window faces.

Hypotheses. H1 held: at overlap 0 the two modes differ by at most 5.96×10−85.96 \times 10^{-8}, and their printed errors and Dice are the same. H2 held: the port places 32, 75, 147 and 845 windows, as the closed form gives; at overlap 0.75 the step is 8 voxels, and along the 64-voxel axis five windows start at 0, 8, 16, 24 and 32. H3 held for both measures: with Gaussian blending the RMS error falls from 5.9×10−25.9 \times 10^{-2} at overlap 0 to 9.3×10−49.3 \times 10^{-4} at 0.75, and the maximum error from 0.98 to 0.014. H4 held for both measures at all three overlaps. The margin is small at 0.25, where the maximum errors are 0.698 and 0.716, and large at 0.5 and 0.75, where the Gaussian RMS errors are 1.36×10−31.36 \times 10^{-3} and 9.34×10−49.34 \times 10^{-4} against 3.99×10−23.99 \times 10^{-2} and 3.09×10−23.09 \times 10^{-2} for constant blending. H5 held: the map’s minimum is float32(10−310^{-3}), so the clamp is active at the default sigma_scale, and the map stays float32.

The thresholded masks follow the same pattern. With Gaussian blending, Dice against the thresholded reference is 0.999546 at overlap 0.5 and 0.999461 at 0.75; with constant blending it is 0.961010, 0.945393 and 0.970326 at 0.25, 0.5 and 0.75.

For constant blending, the printed RMS errors at overlaps 0.25, 0.5 and 0.75 are 3.56×10−23.56 \times 10^{-2}, 3.99×10−23.99 \times 10^{-2} and 3.09×10−23.09 \times 10^{-2}.

The pilot used noise seed 1. Every check and hypothesis held there too, so no outcome differs between the pilot and the first run.

These outcomes concern our port and our fixed filter. A trained network’s error near window borders depends on its receptive field, padding and training, which this example does not model; MONAI’s own blending code, its padding path and its buffered path were not run.

For reference, a call of the real function looks like this. It is written from the 1.6.1 signature and docstring and was not run:

# Not run: MONAI 1.6.1 was not installed. Arguments follow the 1.6.1 signature of
# monai.inferers.sliding_window_inference; `volume` is (N, C, D, H, W), channel-first.
import torch
from monai.inferers import sliding_window_inference

model.eval()
with torch.no_grad():
    pred = sliding_window_inference(
        inputs=volume,
        roi_size=(96, 96, 96),
        sw_batch_size=4,
        predictor=model,
        overlap=0.5,       # default 0.25
        mode="gaussian",   # default "constant"; sigma_scale defaults to 0.125
    )

Ecosystem and Integrations

MONAI depends directly only on PyTorch and NumPy; everything else is optional. Its pyproject.toml lists extras that include ITK, NiBabel, pydicom, OpenSlide, tifffile, cuCIM, CuPy, PyTorch Ignite, MLflow, TensorBoard, ONNX and ONNX Runtime, Polygraphy, transformers, Segment Anything and TorchIO, installed with pip install 'monai[...]'; monai[all] installs almost all of them, except CuPy and nvImageCodec, whose CUDA 12 and 13 builds cannot be chosen automatically (installation guide); pyproject.toml also leaves the hyena extra out of all. Release 1.6.1 added NvImgCodecPydicomReader, a reader for GPU decoding of DICOM images through the nvimgcodec extra. MONAI is distributed through PyPI (monai, plus a weekly monai-weekly preview), conda-forge and Docker images.

The workflow engines build on PyTorch Ignite; the modules document gives PyTorch Lightning and MLflow as other frameworks MONAI fits into, and its experiment tracking and deployment fit the pipelines described in MLOps for Computer Vision. Bundles can carry TorchScript and ONNX exports alongside the PyTorch weights. Tutorials and the Model Zoo live in separate Project MONAI repositories (README). MONAI Label and MONAI Deploy are named here only; neither was read for this article.

For file formats MONAI relies on other libraries: the installation guide names NiBabel for NIfTI files, and ITK and pydicom are among the optional dependencies. ITK and SimpleITK, the DICOM standard and the NIfTI format are separate subjects; MONAI’s RAS-labelled defaults sit beside DICOM’s and ITK’s LPS patient system, a difference set out in Medical Image Preprocessing.

Alternatives

  • nnU-Net is a self-configuring segmentation framework: rather than offering components, it sets the preprocessing, network and training for a dataset by rules and cross-validation, and its inference uses Gaussian-weighted sliding windows with half-patch overlap by default. MONAI includes an nnU-Net app, whose command composition was hardened in 1.6.0 against code injection (GHSA-rghg-q7wp-9767).
  • TorchIO is another library for medical images in PyTorch pipelines; MONAI lists it as an optional dependency, and it is named here only.
  • Plain PyTorch with SimpleITK or NiBabel covers reading, resampling and normalization without a framework (the Medical Image Preprocessing example does these steps with SimpleITK, NiBabel and NumPy), leaving patch sampling, inference stitching and metrics to the user.
  • ITK and SimpleITK are image-analysis toolkits without a deep learning framework: Medical Image Preprocessing uses SimpleITK for reading and resampling, and Medical Image Registration describes SimpleITK’s registration framework.

Versions and Compatibility

As of November 2026, the latest release is MONAI 1.6.1 (27 September 2026), which its release notes describe as primarily a security release, after 1.6.0 (11 June 2026) and 1.5.2 (29 January 2026). Per pyproject.toml at 1.6.1, MONAI requires Python 3.10 or newer, PyTorch 2.8.0 or newer and NumPy 1.24 or newer but below 3.0; the 1.6.0 notes say that testing starts at Python 3.10 and that only PyTorch 2.8 onwards is supported, though older versions will likely still work. The README states that PyTorch support covers the current release and three previous minor versions, and that other dependencies follow the scientific Python SPEC 0 policy where possible. From 1.6.1 the package is defined only in pyproject.toml, and the release notes say that --no-build-isolation is needed when compiling the CUDA extensions against an installed PyTorch.

Security changes in the two releases concern deserialization and injection. In 1.6.0, NumpyReader gained an allow_pickle argument that is off by default, Auto3DSeg algorithm objects are serialized as JSON instead of pickle (pickle can be re-enabled by setting the environment variable MONAI_ALLOW_PICKLE), and the nnU-Net app’s shell commands were hardened. Release 1.6.1 addresses eight further security advisories and restricts CI permissions. Because a bundle stores weights as a pickled state dictionary (modules document), loading a bundle from an untrusted source carries the risks of unpickling; this is our reading of the two documents, not a statement in either.

The example in this article ran with NumPy 2.3.5 and SciPy 1.16.3 against source read at tag 1.6.1; nothing is claimed about later MONAI versions.

Limitations

  • Defaults that need a decision. Sliding-window inference blends with equal weights and 25 % overlap unless told otherwise; DiceLoss includes the background channel and applies no activation; ScaleIntensityRange does not clip; Orientation interprets axis codes with NiBabel’s default labels unless the data are marked LPS. Each is documented, but none is what every user expects.
  • Window placement. The step is truncated to an integer and the last window is shifted back to the border, so windows are not evenly spaced near the far border, and window counts grow quickly with overlap: 32 at overlap 0 and 845 at 0.75 for a 64×128×12864 \times 128 \times 128 volume with 32332^3 windows, as the example prints.
  • Pickle in bundles. Bundle weights are pickled state dictionaries; the release notes’ security fixes show that deserialization has been a source of advisories.
  • Experimental parts. Lazy resampling is still marked experimental in the 1.6.1 documentation.
  • Auto3DSeg’s cost. Auto3DSeg trains every algorithm it generates before ranking and ensembling the checkpoints (modules document).
  • Scope. This article describes software components; nothing in it concerns clinical use.

Related

  • nnU-Net

    A self-configuring framework for biomedical image segmentation that keeps a plain U-Net and sets everything around it, from target spacing, normalization, patch and batch size and network depth to the choice of 2D, 3D and cascaded configurations, from a fingerprint of the training data by fixed parameters, rules and cross-validation.

  • Medical Image Preprocessing

    The steps that turn stored CT and MRI scans into consistent model inputs, from reading DICOM and NIfTI geometry and orientation to resampling images and labels, normalizing intensities by modality and cropping to the foreground, and how each choice changes the result.

  • Dice Coefficient

    Twice the overlap of a predicted and a ground-truth mask divided by their total size, equal to the F1 score over pixels; the standard overlap score of medical image segmentation and, in soft form, a training loss.

  • Hausdorff Distance

    The largest distance from a point of one set to the nearest point of the other, taken in both directions; its directed, percentile, modified and partial (ranked) forms, why filled regions and their boundaries give different values, how libraries extract boundaries and take percentiles, and matching a template under translation with one distance transform.

  • Hounsfield Units

    The scale on which CT scanners report attenuation, fixed at 0 for water and −1000 for air, how scanners are calibrated to it, why tissue values still depend on the X-ray energy and the scanner, and how windowing turns it into a displayed image.

  • U-Net

    A convolutional encoder–decoder for segmentation whose contracting path gathers context and whose symmetric expanding path, fed by concatenated skip connections, restores pixel-accurate boundaries; designed in 2015 for biomedical images with few labels, and since the base of 3D U-Net, Attention U-Net, UNet++, nnU-Net and the denoisers of diffusion models.

References

  1. MONAI Consortium. README.md, Project-MONAI/MONAI at tag 1.6.1 (commit f2ec87a; Apache License 2.0).
  2. MONAI Consortium. Modules (docs/source/modules.md) and Research and Application Highlights (docs/source/applications.md), MONAI 1.6.1 documentation source.
  3. MONAI Consortium. What's new in 1.6.0 (docs/source/whatsnew_1_6.md) and What's new in 1.6.1 (docs/source/whatsnew_1_6_1.md), MONAI 1.6.1 documentation source.
  4. MONAI Consortium. Installation Guide (docs/source/installation.md) and Lazy Resampling (docs/source/lazy_resampling.rst), MONAI 1.6.1 documentation source.
  5. MONAI Consortium. pyproject.toml and CITATION.cff, MONAI 1.6.1 source code.
  6. MONAI Consortium. monai/inferers/utils.py and monai/inferers/inferer.py, MONAI 1.6.1 source code.
  7. MONAI Consortium. monai/data/utils.py, MONAI 1.6.1 source code.
  8. MONAI Consortium. monai/losses/dice.py and monai/losses/utils.py, MONAI 1.6.1 source code.
  9. MONAI Consortium. monai/transforms/spatial/array.py, monai/transforms/intensity/array.py and monai/utils/deprecate_utils.py, MONAI 1.6.1 source code.
  10. MONAI Consortium. MONAI: Medical Open Network for AI, version 1.6.1 (software; the concept DOI represents all versions).
  11. Cardoso, M. J., Li, W., Brown, R., Ma, N., Kerfoot, E., Wang, Y., Murray, B., Myronenko, A., Zhao, C., Yang, D., et al. (2022). MONAI: An open-source framework for deep learning in healthcare. arXiv:2211.02701 (not read).