Vision Systems & Engineering
ITK and SimpleITK
The Insight Toolkit for N-dimensional scientific image analysis and its simplified interface SimpleITK, with ITK's data-flow pipeline, requested regions and streaming, its physical-space image model, IO factories and wrapping, and a port that streams ITK's discrete Gaussian through padded regions.
intermediate
The Insight Toolkit (ITK) is, in its documentation’s words, “an open-source, cross-platform toolkit for N-dimensional scientific image processing, segmentation, and registration”. It is written in C++ and distributed under the Apache 2.0 licence, with copyright held by the Insight Software Consortium for 1999–2019 and by NumFOCUS from 2020. SimpleITK is a simplified programming interface to ITK’s algorithms and data structures, with bindings for C++, Python, R, Java, C#, Lua, Ruby and TCL. Both are common in medical image analysis on this site: Medical Image Preprocessing reorients and resamples with SimpleITK in its example and describes N4 bias correction, and Medical Image Registration describes its registration framework.
This article covers what ITK is built from, chiefly its data-flow pipeline and the region bookkeeping that lets a filter process an image in pieces, and what SimpleITK keeps and leaves out. It was written from ITK v5.4.7 (commit 4c49df9) and SimpleITK v2.5.7, read in their source repositories in November 2026; nothing is claimed about other versions beyond what their release notes say. ITK’s DICOM and NIfTI readers are named in the IO section without a description of the formats. The example is a NumPy and SciPy port of one ITK mechanism, because neither toolkit was installed where this article was written.
Purpose
ITK’s licence page gives its history. The toolkit was first developed under contract to the National Library of Medicine at the National Institutes of Health, and parts of it are derived from VTK and VXL and keep their copyright notices. From version 3.6 it was distributed under the simplified BSD licence, and from ITK 4 under Apache 2.0, after a vote of the Insight Software Consortium board between 15 December 2009 and 14 January 2010. On 26 November 2019 the consortium transferred the copyright to NumFOCUS. The FAQ says ITK may be cited by a paper, the books or the web site, and lists McCormick et al. (2014) and Yoo et al. (2002) as the papers, which we have not read.
One idea shapes both toolkits, and SimpleITK states it in its description of the image: an image is “a set of points on a grid occupying a physical region in space”, not an array. Many image libraries treat images as arrays, and SimpleITK’s documentation names two consequences of doing so: spacing is assumed isotropic, and an image has no location in physical space. The documentation ties ITK’s choice to its original domain, medical imaging, where structures have metric sizes and positions and the spacing between voxels is often not isotropic, most commonly larger along the head-to-foot axis.
SimpleITK’s history page says it was created to make ITK usable by a wider audience. Its initial funding came from the NLM under the American Recovery and Reinvestment Act, its first version was a collaboration of the Mayo Clinic, Kitware, the University of Iowa and NLM’s intramural program, and its first major release was announced in April–May 2017. Since 2019 its development has been supported by the National Institute of Allergy and Infectious Diseases. Its Readme asks users to cite Lowekamp et al. (2013), Yaniv et al. (2018) and Beare et al. (2018), none of which we have read.
Core Capabilities
Generic programming
ITK classes are C++ templates over the pixel type and the image dimension: ImageBase is, in its header’s words, “templated over the dimension of the image”, and a filter is instantiated for an input and output image type. The Python quick start states the consequence for users: filters “are optimized at compile time for each image pixel type and image dimension”, and the Python package detects the type to use from the input image, or takes it from an explicit ttype argument when it cannot be implied. SimpleITK’s conventions page says it uses variable-length vectors for indexes, points and sizes so that non-templated calls can wrap ITK’s templated classes; in our reading, its single image class makes pixel type and dimension run-time properties.
The data-flow pipeline
An ITK program is a graph of data objects (images, meshes) and process objects (sources, filters and mappers, in itkProcessObject.h’s terms), connected input to output. Nothing runs when a filter is connected. A call to Update() on the last filter walks the graph in three passes, in our reading of the method documentation in itkProcessObject.h:
UpdateOutputInformation()travels upstream and, on the way back, lets each filter’sGenerateOutputInformation()set the output’s metadata, such as the spacing a shrinking filter changes.PropagateRequestedRegion()sends the region the caller wants back up the graph. Each filter’sGenerateInputRequestedRegion()decides which part of its input it needs to produce its requested output, andEnlargeOutputRequestedRegion()lets a filter that can only produce whole slices or whole images ask for more.UpdateOutputData()executes, upstream first, only what is out of date.
Three regions describe an image (itkImageBase.h). The LargestPossibleRegion is the whole dataset; the BufferedRegion is the part held in memory; the RequestedRegion is what the pipeline asks a filter to produce. The header writes the nesting as requested buffered largest possible. By default a filter requests its input’s largest possible region. A filter whose output pixel depends on a neighbourhood instead requests a padded copy of its output region, which itkProcessObject.h says is done “to avoid introducing artificial boundary conditions”, and crops the request at the largest possible region, never asking for pixels that do not exist.
Update() has a subtlety that its documentation calls “a confusing side effect”: it updates the current requested region. The first call sets it to the largest possible region; if an upstream change later enlarges the image, a second Update() still produces the old region. UpdateLargestPossibleRegion() always produces the whole image.
Memory is managed by two flags. With ReleaseDataFlag on, a filter’s output is freed once the downstream filter has used it; it defaults to off. ReleaseDataBeforeUpdateFlag, which frees an output that is about to be regenerated, defaults to on for a process object, though some subclasses such as ImageSource change it.
Streaming. Because every filter can be asked for less than the whole image, a pipeline can process an image too large for memory one piece at a time. StreamingImageFilter does this. By default it asks for 10 divisions, and its region splitter, ImageRegionSplitterSlowDimension, cuts the outermost dimension of size greater than 1 (slices of a volume) into pieces of slices for slices and requested pieces, the last piece taking the rest. Because the piece size is rounded up, the splitter can return fewer pieces than requested, and StreamingImageFilter uses the smaller number. For each piece it sets the input’s requested region, propagates it upstream, updates, and copies only the piece itself, not whatever larger region the upstream filters buffered, into its output.
The discrete Gaussian filter
DiscreteGaussianImageFilter shows how a neighbourhood filter cooperates with this machinery, and the example ports it. Its kernel, from GaussianOperator, is Lindeberg’s discrete analogue of the Gaussian, with the modified Bessel function of order and the variance in pixels; Scale Space introduces this kernel and compares derivative kernels built from it with sampled ones. ITK evaluates and with polynomial approximations, which switch form at , and higher orders by a downward recurrence normalized by . It adds coefficients outwards until the kernel’s sum reaches , divides by the sum, and mirrors the coefficients into a symmetric kernel.
The filter’s defaults, from its header, are a variance of 0 and a MaximumError of 0.01 in each dimension, a MaximumKernelWidth of 32 pixels, and UseImageSpacing on, which makes the variance a physical quantity: the variance in pixels along each axis is the given variance divided by that axis’s spacing squared. Out-of-image pixels take the nearest edge value (ZeroFluxNeumannBoundaryCondition). The filter runs as a small internal pipeline of one-dimensional NeighborhoodOperatorImageFilter passes, built in reverse dimension order, so that on a volume the first pass runs along z and the last along x. Its own GenerateInputRequestedRegion() pads the requested region by the kernel radius along every axis and crops it at the largest possible region; inside the chain, each pass pads only along its own axis.
The files disagree with themselves in four places:
itkDiscreteGaussianImageFilter.hxxcomments that the operators are reversed “because the largest dimension will be split slice wise for streaming” and that a streaming filter “forces the mini-pipeline to run in chunks”, and the header documents an internal number of stream divisions with a default of ImageDimension². In the same.hxx,GetInternalNumberOfStreamDivisions()returns 1, and its legacy setter warns that the filter “no longer internally streams”.- The header says the kernel is no wider than MaximumKernelWidth pixels. The loop in
itkGaussianOperator.hxxstops when the count of coefficients on one side, centre included, exceeds that width, so the radius can reach 32 and the kernel taps; the example prints this case. itkGaussianOperator.hsays a warning is given when the kernel is truncated; the.hxxemits the message throughitkDebugMacro.GaussianOperator’s own default MaximumKernelWidth is 30; the discrete Gaussian filter sets 32 on the operators it builds.
The physical-space image model
ImageBase defines the geometry. The origin is the physical position of the pixel whose index is all zeros; the spacing is the pixel size along each axis; the direction is a matrix whose columns give the physical direction of each index axis, the first column for the fastest-moving index. The header expects the columns to form an orthogonal, right-handed system, and adds that this is “not checked nor enforced”. Converting a physical point to an index rounds, with ITK’s RoundHalfIntegerUp, so a point exactly half-way between two pixel centres goes to the higher index.
SimpleITK’s Fundamental Concepts page uses the same four quantities, with size in place of the regions. It says an image’s physical extent starts half a voxel before the origin and ends half a voxel beyond the last voxel, that a new image has origin zero, spacing one and identity direction, and that the units are not stored: it is “up to you the developer to be consistent”. The documentation disagrees with itself on dimensions: conventions.rst says the image class “can contain 2 or 3 dimensional images”, while fundamentalConcepts.rst says the default configuration includes images from 2D up to 5D, and the Readme says SimpleITK supports 2D, 3D and 4D images. Medical Image Preprocessing describes how this model meets DICOM’s patient coordinates, LPS orientation and SimpleITK’s DICOMOrient.
Resampling and interpolation conventions
ResampleImageFilter maps each output point through a transform to the input and interpolates there. Its defaults, as its header states them, are a linear interpolator, “reasonable for ordinary medical images”, nearest-neighbour recommended for label images, a default pixel value of zero for points outside the input, and unit spacing, zero origin and identity direction for the output unless a reference image is given. Several conventions sit in the source rather than the documentation: an image function counts a point as inside the buffer up to half a pixel beyond the first and last pixel centres (itkImageFunction.hxx); linear interpolation there uses the edge pixel; and a value cast to the output pixel type is clamped to that type’s range (CastComponentWithBoundsChecking in itkResampleImageFilter.hxx). Medical Image Preprocessing covers resampling grids, interpolators, label resampling and these defaults in use, and image interpolation the interpolators themselves.
IO factories
ITK reads and writes files through ImageIO classes registered with a factory. SimpleITK’s IO page lists the ones registered in its build: BMP, BioRad, Bruker2dseq, GDCM (DICOM), GE4, GE5, GIPL, HDF5, JPEG, LSM, MINC, MRC, MetaImage (.mha, .mhd), NIfTI (.nii, .nii.gz, .hdr, .img and others), NRRD, PNG, Stimulate, TIFF and VTK. ITK’s FAQ explains the mechanism: to read, ITK asks each registered reader whether it can read the file; to write, it chooses the format from the file name’s extension. An IO module that was not enabled and registered in a C++ build produces the error “There are no registered IO factories”.
Two of the readers handle DICOM and NIfTI. GDCMSeriesFileNames orders the files of a DICOM series by the slices’ 3D positions, computed from Image Orientation and Image Position, falls back to Instance Number, and finally to file names in lexicographical order. Its header suggests calling SetUseSeriesDetails(true) when several volumes are grouped as one series, which adds information such as ProtocolName and SeriesName to the series UID; the same header initializes that flag to true. NiftiImageIO chooses between a NIfTI file’s two orientation encodings, the qform and the sform. In our reading of itkNiftiImageIO.cxx, it prefers the sform when the two matrices agree element by element; otherwise it checks that the sform is affine and, with its column scales removed, orthonormal (it only warns when those scales differ from the voxel sizes), and then prefers the sform when the qform code is unknown, when the sform is labelled scanner-anatomical, or when both codes are set and the two matrices agree after decomposition; an sform that fails the check is orthogonalized and used only with the reader’s permissive setting on; if neither form can be used, it stops with an error that it supports only orthonormal direction cosines.
Wrapping
The itk Python package (pip install itk) offers two styles, according to the Python quick start. Snake-case functions such as itk.median_image_filter(image, radius=2) run at once and return their output. Filter objects such as itk.MedianImageFilter.New(reader) can instead be connected into a pipeline “to stream-process large datasets”, and run only when .Update() is called on the last one. Pixel buffers pass to NumPy as views (itk.array_view_from_image, no copy) or copies (itk.array_from_image). One trap is documented: in the Python dictionary of an image’s metadata, dict(image), the origin, spacing and direction are reversed into NumPy’s z, y, x order, while image.GetSpacing() and itk.dict_from_image keep ITK’s x, y, z order.
SimpleITK wraps ITK through SWIG. Almost every filter exists twice, according to its conventions page: as an object named with ITK’s ImageFilter suffix, such as AbsImageFilter, and as a procedural function without the suffix, such as Abs, which sets parameters and runs in one call. Pixel access is in x, y, z order, while the NumPy arrays it exchanges are ordered z, y, x, as Medical Image Preprocessing’s example shows. Masks default to uint8 images with values 0 and 1, matrices are passed as row-major vectors, a region is given as its index followed by its size, and ImageRegistrationMethod accepts only float32 and float64 images. The FAQ says SimpleITK supports most ITK image filters and the IO and registration frameworks, exposes fewer settings, and omits “the pipeline architecture, spatial objects framework, point sets, and the mesh framework”. In our reading, a SimpleITK filter call therefore runs at once on an image held in memory, and streaming a chain of filters needs ITK’s pipeline.
Minimal Example
Neither ITK nor SimpleITK could be installed where this article was written, so the example ports the mechanism of the streaming section to NumPy 2.3.5 and SciPy 1.16.3 and checks the port’s bookkeeping. The outcomes test our port of ITK v5.4.7’s code at commit 4c49df9, not ITK. The ported files are itkGaussianOperator.hxx (the kernel and its Bessel approximations), itkDiscreteGaussianImageFilter.hxx (variance per axis, pass order, padding), itkStreamingImageFilter.hxx (the piece loop), itkImageRegionSplitterSlowDimension.cxx and the padding of itkNeighborhoodOperatorImageFilter.hxx; each pass applies the kernel with scipy.ndimage.correlate1d(mode="nearest"), SciPy’s edge replication. Region padding and cropping are our own interval arithmetic.
Data. A synthetic volume of 64 slices of 96 × 96 voxels (NumPy shape z, y, x; ITK size 96, 96, 64) with spacing 0.8 × 0.8 × 3 mm, holding three ellipsoids defined in millimetres with values 100, 300 and 600 on a background of 0, plus Gaussian noise with standard deviation 20. The filter uses a variance of 4 mm² with UseImageSpacing on, so the pixel variances are 6.25 in x and y and 0.444 in z.
Runs. W filters the whole volume. S streams it in the 10 pieces the splitter returns: for each piece the script propagates the requested region backwards through the x, y and z passes, each padding along its own axis and cropping at the volume, then runs the three passes forwards on the buffered input and keeps the piece. N is naive tiling: each piece is filtered alone, so edge replication happens at every piece border.
Checks and hypotheses. C-data checks the generated volume; C-kernel compares the port’s coefficients, before normalization, with SciPy’s special.ive (which computes ) at both pixel variances the run uses, one on each side of the polynomial switch at 3.75, and checks that the normalized kernels sum to 1; C-boundary checks edge replication against the 3 × 5 corner example printed in itkZeroFluxNeumannBoundaryCondition.h and correlate1d against a hand loop. The hypotheses were:
- H1: S equals W bit for bit.
- H2: N differs from W exactly on the slices within the z radius of an interior seam, and nowhere else.
- H3: 10 requested pieces give nine of 7 slices and one of 1; 12 give 11 pieces, ten of 6 and one of 4.
- H4: with
UseImageSpacingon, the z radius is smaller than the x and y radii; with it off, the three are equal; for variances of 4 and 400 mm². - H5: at 400 mm², the radius stops at the cap of 32 (65 taps) wherever the pixel variance is 400 or 625, and stays below it along z with spacing on (pixel variance 44.4).
- H6: S’s largest buffered input is of the volume, N’s is , and W’s is the whole volume.
Before the run. The hypotheses were written before any script ran. A planning script then computed the same quantities independently of the port’s region code and splitter, with the exact Bessel kernel from special.ive for the radii, a separate transcription of ITK’s polynomials for the filtering, the piece table written out by hand and plain array slicing, on the same data and both seeds. It confirmed H1, H2, H4, H5 and H6 before the pilot, so the example is an illustration: it checks that our port’s bookkeeping agrees with an independent computation, not an experiment whose outcome was unknown. Four tolerances were first written after that script had printed the values they bound: the noise standard deviation within 0.005 of 20 (relative), the noise mean within 0.1, C-kernel’s difference at most and C-boundary’s at most ; the kernel-sum bar of was written before. H6’s per-piece prediction was corrected after the planning script: its first form said that pieces 1 to 8 buffer 11 slices, and the script printed 10 for piece 8, whose padded request is cropped at the last slice; the prediction became 9, 11 for pieces 1 to 7, 10 and 3.
The batch plan’s description of the example was extended, with the editor’s approval before the first run, by the second C-kernel variance, the header-example check, the per-stage region print, the stopping-margin column of the radius table, the 400 mm² variance and its cap hypothesis H5, hypothesis H6, the 12-piece table’s last piece of 4 and the padding by the outer filter as well as by each pass; the data parameters and seeds are our choices. A pilot with noise seed 1 and the first run with seed 0, shown below, gave the same outcome for every check and hypothesis; they differ only in the noise statistics (sd/20 − 1 = −0.00131 and mean −0.0304 in the pilot), in C-boundary’s difference ( in the pilot) and in max (25.73 in the pilot). The script runs in well under a second, and two runs printed identical output.
import os
for v in ("OPENBLAS_NUM_THREADS", "OMP_NUM_THREADS", "MKL_NUM_THREADS"):
os.environ[v] = "1" # one BLAS thread, set before NumPy is imported
import math
import sys
import numpy as np
import scipy
from scipy import ndimage, special
# Ported (modified) from ITK v5.4.7, commit 4c49df934c316ff87dbbd971b89e63df6bbd1836:
# itkGaussianOperator.hxx, itkDiscreteGaussianImageFilter.hxx, itkStreamingImageFilter.hxx,
# itkImageRegionSplitterSlowDimension.cxx and itkNeighborhoodOperatorImageFilter.hxx.
# Copyright NumFOCUS. Licensed under the Apache License, Version 2.0; modified (ported to Python).
# https://github.com/InsightSoftwareConsortium/ITK/blob/4c49df934c316ff87dbbd971b89e63df6bbd1836/LICENSE
# Region padding and cropping below are our own arithmetic on intervals, not ported from ITK.
SEED = int(sys.argv[1]) if len(sys.argv) > 1 else 0
print("NumPy", np.__version__, "| SciPy", scipy.__version__, "| noise seed", SEED)
# ---- GaussianOperator: Lindeberg's kernel from ITK's polynomial Bessel approximations ----
def bessel_i0(y):
d = abs(y)
if d < 3.75:
m = (y / 3.75) ** 2
return 1.0 + m * (3.5156229 + m * (3.0899424 + m * (1.2067492 + m * (0.2659732
+ m * (0.360768e-1 + m * 0.45813e-2)))))
m = 3.75 / d
return (math.exp(d) / math.sqrt(d)) * (0.39894228 + m * (0.1328592e-1 + m * (0.225319e-2
+ m * (-0.157565e-2 + m * (0.916281e-2 + m * (-0.2057706e-1 + m * (0.2635537e-1
+ m * (-0.1647633e-1 + m * 0.392377e-2))))))))
def bessel_i1(y):
d = abs(y)
if d < 3.75:
m = (y / 3.75) ** 2
acc = d * (0.5 + m * (0.87890594 + m * (0.51498869 + m * (0.15084934 + m * (0.2658733e-1
+ m * (0.301532e-2 + m * 0.32411e-3))))))
else:
m = 3.75 / d
acc = 0.2282967e-1 + m * (-0.2895312e-1 + m * (0.1787654e-1 - m * 0.420059e-2))
acc = 0.39894228 + m * (-0.3988024e-1 + m * (-0.362018e-2 + m * (0.163801e-2
+ m * (-0.1031555e-1 + m * acc))))
acc *= math.exp(d) / math.sqrt(d)
return -acc if y < 0.0 else acc
def bessel_in(n, y): # n >= 2: downward recurrence, normalized by I0
if y == 0.0:
return 0.0
toy, qip, acc, qi = 2.0 / abs(y), 0.0, 0.0, 1.0
for j in range(2 * (n + int(math.sqrt(40.0 * n))), 0, -1):
qim = qip + j * toy * qi
qip, qi = qi, qim
if abs(qi) > 1.0e10:
acc, qi, qip = acc * 1.0e-10, qi * 1.0e-10, qip * 1.0e-10
if j == n:
acc = qip
acc *= bessel_i0(y) / qi
return -acc if (y < 0.0 and n & 1) else acc
def gaussian_coefficients(t, max_error=0.01, max_width=32):
"""GenerateCoefficients: returns (symmetric kernel, raw one-sided e^-t I_n(t), stop margin)."""
et = math.exp(-t)
cap = 1.0 - max_error
c = [et * bessel_i0(t), et * bessel_i1(t)]
s = c[0] + 2.0 * c[1]
i = 2
while s < cap:
c.append(et * bessel_in(i, t))
s += 2.0 * c[i]
if c[i] <= 0.0:
break # failsafe
if len(c) > max_width:
break # truncated at MaximumKernelWidth (debug message only)
i += 1
half = np.array(c) / s
return np.concatenate([half[:0:-1], half]), np.array(c), s - cap
# ---- DiscreteGaussianImageFilter settings: variance in mm^2 divided by spacing^2 (UseImageSpacing on)
def pixel_variances(variance, spacing_xyz, use_spacing=True):
return [variance / (s * s) if use_spacing else variance for s in spacing_xyz]
# ---- ImageRegionSplitterSlowDimension and StreamingImageFilter's piece count ----
def splitter_pieces(size_xyz, requested):
"""Pieces of the outermost dimension of size > 1, as (start, size) along that dimension."""
axis = len(size_xyz) - 1
while size_xyz[axis] <= 1:
axis -= 1
rng = float(size_xyz[axis])
per = math.ceil(rng / max(1, requested))
n = min(requested, math.ceil(rng / per)) # StreamingImageFilter: min(requested, splitter)
per = math.ceil(rng / n) # GetSplitInternal recomputes from n
last = math.ceil(rng / per) - 1
pieces = [(i * per, per) for i in range(last)]
pieces.append((last * per, size_xyz[axis] - last * per))
return axis, pieces
# ---- Regions in ITK order: index (x, y, z) and size (x, y, z); NumPy slices are reversed ----
def pad_crop(index, size, radius, largest_size):
lo = [max(i - r, 0) for i, r in zip(index, radius)]
hi = [min(i + n + r, L) for i, n, r, L in zip(index, size, radius, largest_size)]
return lo, [h - l for l, h in zip(lo, hi)]
def to_slices(index, size):
return tuple(slice(i, i + n) for i, n in zip(index[::-1], size[::-1])) # (z, y, x)
def filter_region(volume, out_index, out_size, kernels_xyz, largest):
"""One update of the filter for an output requested region (index, size in x, y, z).
Requested regions are propagated backwards: the last stage (x) pads x, the middle (y)
pads y, the first (z) pads z, each cropped at the largest possible region. The data then
flow forwards; ZeroFluxNeumann (edge replication) applies at each buffer's edge."""
radii = [(len(k) - 1) // 2 for k in kernels_xyz]
stage_axes = [2, 1, 0] # ITK dims in pass order: z, y, x
req = [None, None, None]
idx, siz = out_index, out_size
for st in (2, 1, 0): # backwards from the last stage
rad = [0, 0, 0]
rad[stage_axes[st]] = radii[stage_axes[st]]
req[st] = ((idx, siz), pad_crop(idx, siz, rad, largest)) # (output, input) regions
idx, siz = req[st][1]
outer_in = pad_crop(out_index, out_size, radii, largest) # DiscreteGaussian's own request
buf_index, buf_size = req[0][1]
data = volume[to_slices(buf_index, buf_size)]
for st in range(3): # forwards: z, y, x passes
d = stage_axes[st]
data = ndimage.correlate1d(data, kernels_xyz[d], axis=2 - d, mode="nearest")
(o_idx, o_siz), (i_idx, _) = req[st]
crop = tuple(slice(o - i, o - i + n) for o, i, n in zip(o_idx[::-1], i_idx[::-1], o_siz[::-1]))
data = data[crop]
return data, outer_in, req
# ---- Data: synthetic volume, NumPy (z, y, x) = (64, 96, 96), spacing (x, y, z) = (0.8, 0.8, 3) mm
SPACING = (0.8, 0.8, 3.0)
SIZE = (96, 96, 64) # ITK order (x, y, z)
z, y, x = np.meshgrid(np.arange(SIZE[2]) * SPACING[2], np.arange(SIZE[1]) * SPACING[1],
np.arange(SIZE[0]) * SPACING[0], indexing="ij")
clean = np.zeros(SIZE[::-1])
for (cx, cy, cz), (ax, ay, az), value in [((38, 38, 96), (32, 26, 80), 100.0),
((28, 36, 70), (8, 10, 30), 300.0),
((48, 40, 120), (10, 8, 24), 600.0)]:
clean[((x - cx) / ax) ** 2 + ((y - cy) / ay) ** 2 + ((z - cz) / az) ** 2 <= 1.0] = value
vol = clean + np.random.default_rng(SEED).normal(0.0, 20.0, clean.shape)
vals, counts = np.unique(clean, return_counts=True)
res = vol - clean
print("data:", vol.shape, vol.dtype, "| values", vals.tolist(), "counts", counts.tolist(),
"| noise sd/20 - 1 = %+.5f (tol 0.005), mean %+.4f (tol 0.1)" % (res.std() / 20 - 1, res.mean()))
# ---- C-kernel: ported coefficients against scipy.special.ive, both polynomial branches ----
print("\nC-kernel (pre-normalization e^-t I_n(t) against ive; tol 1e-7; sum tol 1e-12)")
for t in (6.25, 4.0 / 9.0):
k, raw, _ = gaussian_coefficients(t)
ref = special.ive(np.arange(len(raw)), t)
print(" t = %.4f: radius %d, max |port - ive| %.3e, |sum - 1| %.3e"
% (t, len(raw) - 1, np.abs(raw - ref).max(), abs(k.sum() - 1.0)))
# ---- C-boundary: (a) the documented ZeroFluxNeumann example, (b) correlate1d vs a hand loop ----
corner = np.array([[1, 2, 3, 4, 5], [3, 3, 5, 5, 6], [4, 4, 6, 7, 8]])
rows = np.clip(np.arange(-2, 3), 0, 2) # 5 x 7 neighbourhood centred on the 2
cols = np.clip(np.arange(1 - 3, 1 + 4), 0, 4)
documented = np.array([[1, 1, 1, 2, 3, 4, 5], [1, 1, 1, 2, 3, 4, 5], [1, 1, 1, 2, 3, 4, 5],
[3, 3, 3, 3, 5, 5, 6], [4, 4, 4, 4, 6, 7, 8]])
print("\nC-boundary (a) header example reproduced:", np.array_equal(corner[np.ix_(rows, cols)], documented))
kx = gaussian_coefficients(6.25)[0]
kz = gaussian_coefficients(4.0 / 9.0)[0]
blk = vol[:5, :6, :7].copy()
worst = 0.0
for axis in range(3):
for k in (kz, kx):
r, n = (len(k) - 1) // 2, blk.shape[axis]
ref = np.zeros_like(blk)
for i in range(n):
acc = 0.0
for j in range(-r, r + 1):
acc = acc + k[j + r] * np.take(blk, min(max(i + j, 0), n - 1), axis=axis)
sl = [slice(None)] * 3
sl[axis] = i
ref[tuple(sl)] = acc
worst = max(worst, np.abs(ndimage.correlate1d(blk, k, axis=axis, mode="nearest") - ref).max())
print("C-boundary (b) correlate1d(mode='nearest') vs clamped hand loop: max |diff| %.3e (tol 1e-12)" % worst)
# ---- M5: kernel radius per axis ----
print("\nM5 kernel radius (width) per axis x, y, z; stop margin = sum - 0.99")
for var in (4.0, 400.0):
for on in (True, False):
parts = []
for ax_name, t in zip("xyz", pixel_variances(var, SPACING, on)):
k, raw, margin = gaussian_coefficients(t)
parts.append("%s %2d (%2d) %+.2e" % (ax_name, len(raw) - 1, len(k), margin))
print(" variance %5.0f mm2, UseImageSpacing %-3s | %s" % (var, "on" if on else "off", " | ".join(parts)))
# ---- M1: piece tables ----
print("\nM1 piece tables (start, size) along ITK dimension 2 = NumPy axis 0")
for req_n in (10, 12):
axis, pcs = splitter_pieces(SIZE, req_n)
print(" requested %2d -> %2d pieces on dim %d: %s" % (req_n, len(pcs), axis, pcs))
# ---- Runs W, S, N with variance 4 mm^2, UseImageSpacing on ----
kernels = [gaussian_coefficients(t)[0] for t in pixel_variances(4.0, SPACING)] # x, y, z
rz = (len(kernels[2]) - 1) // 2
W, _, _ = filter_region(vol, [0, 0, 0], list(SIZE), kernels, SIZE)
axis, pieces = splitter_pieces(SIZE, 10)
S = np.empty_like(W)
N = np.empty_like(W)
print("\nM4 per piece of S: output z range, buffered input z range (slices, fraction of volume)")
fracs = []
for p, (a, n) in enumerate(pieces):
idx, siz = [0, 0, a], [SIZE[0], SIZE[1], n]
out, outer_in, req = filter_region(vol, idx, siz, kernels, SIZE)
S[to_slices(idx, siz)] = out
alone, _, _ = filter_region(vol[to_slices(idx, siz)], [0, 0, 0], siz, kernels, siz)
N[to_slices(idx, siz)] = alone
buf = req[0][1]
assert buf == outer_in # z-stage input = the outer request
fracs.append(math.prod(buf[1]) / W.size)
print(" piece %d: out [%2d, %2d) in [%2d, %2d) %2d slices %.6f"
% (p, a, a + n, buf[0][2], buf[0][2] + buf[1][2], buf[1][2], fracs[-1]))
if p == 1:
chain = req
print(" largest buffered fraction: S %.6f | N %.6f | W %.6f"
% (max(fracs), max(n for _, n in pieces) / SIZE[2], 1.0))
print(" piece 1 requested regions, ITK (index; size) in x, y, z order:")
for st, name in zip((2, 1, 0), ("x stage", "y stage", "z stage")):
(oi, os_), (ii, is_) = chain[st]
print(" %s: output (%s; %s) <- input (%s; %s)" % (name, oi, os_, ii, is_))
print("\nM2 max |S - W| %.3e, array_equal(S, W) %s | max |N - W| %.3e"
% (np.abs(S - W).max(), np.array_equal(S, W), np.abs(N - W).max()))
pred = np.zeros(SIZE[2], bool)
for a, n in pieces:
for zz in range(a, a + n):
pred[zz] = (a > 0 and zz - a < rz) or (a + n < SIZE[2] and a + n - 1 - zz < rz)
pmask = np.broadcast_to(pred[:, None, None], W.shape)
differ = N != W
print("M3 r_z = %d | voxels N != W %d, predicted %d, predicted but equal %d, differing not predicted %d"
% (rz, differ.sum(), pmask.sum(), (pmask & ~differ).sum(), (differ & ~pmask).sum()))
print(" differing slices:", np.flatnonzero(differ.any(axis=(1, 2))).tolist())
NumPy 2.3.5 | SciPy 1.16.3 | noise seed 0
data: (64, 96, 96) float64 | values [0.0, 100.0, 300.0, 600.0] counts [444580, 135881, 5208, 4155] | noise sd/20 - 1 = +0.00076 (tol 0.005), mean +0.0248 (tol 0.1)
C-kernel (pre-normalization e^-t I_n(t) against ive; tol 1e-7; sum tol 1e-12)
t = 6.2500: radius 7, max |port - ive| 2.261e-09, |sum - 1| 3.331e-16
t = 0.4444: radius 2, max |port - ive| 1.457e-08, |sum - 1| 0.000e+00
C-boundary (a) header example reproduced: True
C-boundary (b) correlate1d(mode='nearest') vs clamped hand loop: max |diff| 1.421e-14 (tol 1e-12)
M5 kernel radius (width) per axis x, y, z; stop margin = sum - 0.99
variance 4 mm2, UseImageSpacing on | x 7 (15) +6.48e-03 | y 7 (15) +6.48e-03 | z 2 ( 5) +7.49e-03
variance 4 mm2, UseImageSpacing off | x 5 (11) +2.37e-03 | y 5 (11) +2.37e-03 | z 5 (11) +2.37e-03
variance 400 mm2, UseImageSpacing on | x 32 (65) -2.06e-01 | y 32 (65) -2.06e-01 | z 17 (35) +1.16e-03
variance 400 mm2, UseImageSpacing off | x 32 (65) -1.07e-01 | y 32 (65) -1.07e-01 | z 32 (65) -1.07e-01
M1 piece tables (start, size) along ITK dimension 2 = NumPy axis 0
requested 10 -> 10 pieces on dim 2: [(0, 7), (7, 7), (14, 7), (21, 7), (28, 7), (35, 7), (42, 7), (49, 7), (56, 7), (63, 1)]
requested 12 -> 11 pieces on dim 2: [(0, 6), (6, 6), (12, 6), (18, 6), (24, 6), (30, 6), (36, 6), (42, 6), (48, 6), (54, 6), (60, 4)]
M4 per piece of S: output z range, buffered input z range (slices, fraction of volume)
piece 0: out [ 0, 7) in [ 0, 9) 9 slices 0.140625
piece 1: out [ 7, 14) in [ 5, 16) 11 slices 0.171875
piece 2: out [14, 21) in [12, 23) 11 slices 0.171875
piece 3: out [21, 28) in [19, 30) 11 slices 0.171875
piece 4: out [28, 35) in [26, 37) 11 slices 0.171875
piece 5: out [35, 42) in [33, 44) 11 slices 0.171875
piece 6: out [42, 49) in [40, 51) 11 slices 0.171875
piece 7: out [49, 56) in [47, 58) 11 slices 0.171875
piece 8: out [56, 63) in [54, 64) 10 slices 0.156250
piece 9: out [63, 64) in [61, 64) 3 slices 0.046875
largest buffered fraction: S 0.171875 | N 0.109375 | W 1.000000
piece 1 requested regions, ITK (index; size) in x, y, z order:
x stage: output ([0, 0, 7]; [96, 96, 7]) <- input ([0, 0, 7]; [96, 96, 7])
y stage: output ([0, 0, 7]; [96, 96, 7]) <- input ([0, 0, 7]; [96, 96, 7])
z stage: output ([0, 0, 7]; [96, 96, 7]) <- input ([0, 0, 5]; [96, 96, 11])
M2 max |S - W| 0.000e+00, array_equal(S, W) True | max |N - W| 2.393e+01
M3 r_z = 2 | voxels N != W 322560, predicted 322560, predicted but equal 0, differing not predicted 0
differing slices: [5, 6, 7, 8, 12, 13, 14, 15, 19, 20, 21, 22, 26, 27, 28, 29, 33, 34, 35, 36, 40, 41, 42, 43, 47, 48, 49, 50, 54, 55, 56, 57, 61, 62, 63]
Data and kernel checks. The data check held: the volume has the planned shape and type, its noise-free values are exactly the four planned levels, each present, and the noise’s standard deviation and mean are within their tolerances. C-kernel held at both pixel variances: the port’s coefficients differ from SciPy’s by at most , well inside , so the polynomial approximations are accurate at both of the run’s variances, and the normalized kernels sum to 1 within . Both C-boundary checks held: index clamping reproduces the header’s documented neighbourhood exactly, corners included, and correlate1d with mode="nearest" matches the hand loop to , also with the x kernel of radius 7 on a block only 5 to 7 voxels long, where replication reaches several voxels beyond the edge.
Kernel radii. H4 held under all four conditions. With spacing on, a variance of 4 mm² gives radius 7 along x and y but 2 along the 3 mm z axis, because dividing by the spacing squared makes the z pixel variance times smaller than the in-plane one (0.444 against 6.25); at 400 mm² the z radius is 17 against 32. With spacing off, the variance is read in pixels and the three radii are equal: 5 at 4 and 32 at 400. H5 held: at 400 mm², every axis whose pixel variance is 400 or 625 stops at radius 32, a kernel of 65 taps, while z with spacing on stops at 17 by the error rule. The margin column is the kernel sum minus 0.99 when the loop stopped; it is negative exactly where the width cap, not the error rule, ended the loop, so those kernels are normalized after dropping more than the 1% MaximumError allows. In our reading, a variance meant in pixels but given while UseImageSpacing is still on is silently divided by the spacing squared, and on anisotropic data each axis is then smoothed by a different amount.
Pieces. H3 held for both requests. Ten requested divisions of 64 slices give pieces of , and 64 slices need only nine of them plus a last piece of a single slice. Twelve requested give pieces of , which cover the volume in pieces, so StreamingImageFilter runs 11, not 12, with a last piece of 4. Uneven pieces are therefore normal: the splitter keeps pieces equal and puts the remainder last.
Streaming against tiling. H1 held: the streamed result equals the whole-volume result bit for bit. The region chain for piece 1 shows why. The x and y passes ask for the piece itself, because the piece already spans the whole volume in x and y and their padding is cropped away, while the z pass, which runs first, asks for slices 5 to 15 for an output of slices 7 to 13: two extra slices on each side, the z radius. Every output voxel of the piece is therefore computed from the same input values, in the same order, as in the whole-volume run, and edge replication only happens where the padded request was cropped at the true top and bottom of the volume.
H2 held: naive tiling changed exactly the 35 predicted slices, the two on each side of every interior seam plus the single-slice last piece and the two slices before it, 322,560 voxels, and no voxel elsewhere. The errors reach 23.93 grey levels, about a quarter of the body ellipsoid’s value of 100. By construction, a voxel next to a seam sees its own slice replicated in place of the slices beyond it, so the error depends on how much the data change across the seam, noise included. H6 held: the largest buffered input of a streamed piece is of the volume, against for naive tiling and the whole volume for W; the end pieces buffer less (9, 10 and 3 slices) because their requests are cropped at the volume. The padding costs four slices per interior piece; with more divisions the pieces shrink while the padding does not, so the extra reads grow as a fraction of each piece (our arithmetic from the construction). U-Net’s overlap-tile strategy pays the same kind of margin for a network instead of a filter.
Ecosystem and Integrations
Several site examples use SimpleITK 2.5.7 directly: Medical Image Preprocessing for orientation codes and resampling, nnU-Net for writing the volumes of its planning example, and Neuroimaging Analysis for bias correction with SimpleITK’s N4 filter. Region growing describes ITK 5.4.0’s region growing filters, ConnectedThreshold, NeighborhoodConnected, ConfidenceConnected and IsolatedConnected. Medical Image Registration lists elastix, an ITK-based registration toolbox with Python access through ITKElastix; SimpleITK’s FAQ adds that its binary distributions do not include SimpleElastix, which needs a SuperBuild with SimpleITK_USE_ELASTIX turned on.
The ITK Python package converts images to and from NumPy arrays, xarray.DataArray and VTK’s vtkImageData while keeping their metadata, and to napari layers through the separate itk-napari-conversion package, according to the Python quick start. The FAQ says that ITK does not need VTK in its default configuration, that optional modules such as ITKVTKGlue do, and that other interpreted languages are reached through SimpleITK and ITK-Wasm. SimpleITK is installed with pip install simpleitk or from conda-forge, according to its getting-started page.
Alternatives
This article does not rank the alternatives.
- scikit-image works on plain NumPy arrays, so an image carries no spacing, origin or direction: the array model that SimpleITK’s documentation contrasts with its own. Its
gaussianreplicates edges by default, as its article shows. - OpenCV is, as the scikit-image article describes it, a C++ library with Python bindings that also works on NumPy arrays; its arrays likewise carry no physical-space metadata.
- MONAI, from the MONAI Consortium, is a further open-source option for medical images; the Hausdorff distance article examines one of its metrics.
- VTK is the visualization toolkit from which parts of ITK are derived; ITK’s FAQ points to VTK’s readers for formats ITK lacks.
- ANTs and elastix are registration toolkits; Medical Image Registration describes both, elastix as ITK-based.
Versions and Compatibility
- ITK 5.4.7 is the latest release as of November 2026: tagged on 21 July 2026 and published on 10 August 2026. Its release notes describe a maintenance release whose highlights include 5D NRRD reading and writing, 32-bit integer TIFF support, faster diffusion tensor reconstruction, and Python packages that now use the
Poolthread backend by default instead of TBB. - ITK 6.0 exists as pre-releases; v6.0b02 was published on 25 March 2026. Nothing in this article was read from it.
- SimpleITK 2.5.7 (30 October 2026) is the latest release; v3.0.0b1 (18 May 2026) and a tag named “latest” (13 November 2026) are pre-releases on its releases page.
- Licences. ITK is Apache 2.0, with third-party code under its own compatible licences listed on the licence page. SimpleITK’s NOTICE names the Insight Software Consortium (2010–2019) and NumFOCUS (2020) as copyright holders, under Apache 2.0.
Limitations
- SimpleITK has no pipeline. Its FAQ lists the pipeline architecture, spatial objects, point sets and meshes as omitted from ITK, so it cannot stream an image through a filter chain; the ITK Python package’s filter objects can.
- Streaming saves memory only where filters cooperate. A filter that keeps ProcessObject’s default request, the largest possible region, makes every piece read the whole input (our reading of
itkProcessObject.h), and neighbourhood filters reread their padding for every piece. - Kernel size is capped. The discrete Gaussian kernel stops at radius 32 even when MaximumError asks for more, with only a debug message; the example shows the cap at 400 mm².
- The geometry is trusted, not checked.
ImageBasedoes not check that the direction matrix is orthogonal and right-handed, and SimpleITK stores no units. - Index orders differ between layers. ITK and SimpleITK index x, y, z, NumPy arrays z, y, x, and the
itkpackage’s metadata dictionary reverses the spatial entries to match NumPy, while its getters do not. - The documentation and the code disagree in places, as the discrete Gaussian’s comments on internal streaming and the documented number of SimpleITK image dimensions show; the source of the version in use is the safer guide.
Related
- Region Growing
Segmenting an image by growing regions from seed pixels for as long as a homogeneity criterion holds, as in seeded region growing, or by splitting non-uniform quadtree blocks and merging similar neighbours.
- Scale Space
The family of Gaussian-smoothed copies of an image indexed by a continuous scale parameter, with the axioms that single out the Gaussian and the normalized derivatives that select a characteristic scale for each image structure.
- Gaussian Filter
A linear smoothing filter that replaces each pixel by an average of its neighbors weighted by a Gaussian, whose standard deviation sets the scale of the detail removed.
- scikit-image
The Python image processing library that works on NumPy arrays, with its modules, its conventions for data types and coordinates, its versions and licence, and an example that checks those conventions on a synthetic image.
- OpenCV
The open-source C++ library of computer vision and image processing functions, with Python bindings that work on NumPy arrays; its modules, conventions, versions from 1.0 to 5.0, contrib modules and licence.
- nnU-Net
A self-configuring framework for biomedical image segmentation that keeps a plain U-Net and sets everything around it, from target spacing, normalization, patch and batch size and network depth to the choice of 2D, 3D and cascaded configurations, from a fingerprint of the training data by fixed parameters, rules and cross-validation.
References
- ITK contributors (2026). ITK Software Licenses, Documentation/docs/licenses.md, ITK v5.4.7 source code.
- ITK contributors (2026). Welcome to ITK's documentation, Documentation/docs/index.md, ITK v5.4.7 source code.
- ITK contributors (2026). Frequently asked questions, Documentation/docs/learn/faq.md, ITK v5.4.7 source code.
- ITK contributors (2026). Python Quick Start, Documentation/docs/learn/python_quick_start.md, ITK v5.4.7 source code.
- ITK contributors (2026). Pipeline and image classes, Modules/Core/Common/include/itkProcessObject.h and itkImageBase.h, ITK v5.4.7 source code.
- ITK contributors (2026). Streaming and region splitting, Modules/Core/Common/include/itkStreamingImageFilter.hxx, itkImageRegionSplitterBase.h and src/itkImageRegionSplitterSlowDimension.cxx, ITK v5.4.7 source code.
- ITK contributors (2026). Discrete Gaussian filter, Modules/Filtering/Smoothing/include/itkDiscreteGaussianImageFilter.h and .hxx, ITK v5.4.7 source code.
- ITK contributors (2026). Gaussian operator, Modules/Core/Common/include/itkGaussianOperator.h and .hxx, ITK v5.4.7 source code.
- ITK contributors (2026). Neighbourhood filtering and boundary conditions, Modules/Filtering/ImageFilterBase/include/itkNeighborhoodOperatorImageFilter.hxx and Modules/Core/Common/include/itkZeroFluxNeumannBoundaryCondition.h, ITK v5.4.7 source code.
- ITK contributors (2026). Resampling and interpolation, Modules/Filtering/ImageGrid/include/itkResampleImageFilter.h and .hxx, Modules/Core/ImageFunction/include/itkImageFunction.hxx, itkLinearInterpolateImageFunction.hxx and itkNearestNeighborInterpolateImageFunction.h, ITK v5.4.7 source code.
- ITK contributors (2026). DICOM series ordering and NIfTI reading, Modules/IO/GDCM/include/itkGDCMSeriesFileNames.h and Modules/IO/NIFTI/src/itkNiftiImageIO.cxx, ITK v5.4.7 source code.
- ITK contributors (2026). LICENSE (Apache License 2.0), ITK v5.4.7 source code.
- ITK (2026). ITK 5.4.7 release notes, GitHub release, published 10 August 2026 (read through the GitHub releases API).
- SimpleITK contributors (2026). About and History, docs/source/about.rst and history.rst, SimpleITK v2.5.7 source code.
- SimpleITK contributors (2026). Fundamental Concepts and Common Conventions, docs/source/fundamentalConcepts.rst and conventions.rst, SimpleITK v2.5.7 source code.
- SimpleITK contributors (2026). Frequently Asked Questions, docs/source/faq.rst, SimpleITK v2.5.7 source code.
- SimpleITK contributors (2026). Getting Started, docs/source/gettingStarted.rst, SimpleITK v2.5.7 source code.
- SimpleITK contributors (2026). Reading and Writing for Images and Transforms, docs/source/IO.rst, SimpleITK v2.5.7 source code.
- SimpleITK contributors (2026). Readme.md, LICENSE and NOTICE, SimpleITK v2.5.7 source code.
- SimpleITK (2026). Releases v2.5.7 (30 October 2026), v3.0.0b1 and latest (pre-releases), GitHub releases (read through the GitHub releases API).
- McCormick, M., Liu, X., Jomier, J., Marion, C. & Ibanez, L. (2014). ITK: Enabling Reproducible Research and Open Science. Frontiers in Neuroinformatics, 8, 13. (not read)
- Yoo, T. S., Ackerman, M. J., Lorensen, W. E., Schroeder, W., Chalana, V., Aylward, S., Metaxas, D. & Whitaker, R. (2002). Engineering and Algorithm Design for an Image Processing API: A Technical Report on ITK – The Insight Toolkit. Proceedings of Medicine Meets Virtual Reality, IOS Press, 586–592. (not read)
- Lowekamp, B. C., Chen, D. T., Ibáñez, L. & Blezek, D. (2013). The Design of SimpleITK. Frontiers in Neuroinformatics, 7, 45. (not read)
- Yaniv, Z., Lowekamp, B. C., Johnson, H. J. & Beare, R. (2018). SimpleITK Image-Analysis Notebooks: a Collaborative Environment for Education and Reproducible Research. Journal of Digital Imaging, 31(3), 290–303. (not read)
- Beare, R., Lowekamp, B. C. & Yaniv, Z. (2018). Image Segmentation, Registration and Characterization in R with SimpleITK. Journal of Statistical Software, 86(8). (not read)