Mathematical Morphology

Mathematical morphology is a framework for analyzing and processing images in terms of shape. Instead of weighting and summing pixel values, as linear filters do, it probes an image with a small reference shape, the structuring element, and asks set-theoretic questions: does the shape fit inside the object here, does it touch it? Two basic operators, erosion and dilation, answer these questions, and almost everything else in the field (opening, closing, morphological gradients, top-hats, skeletons, reconstruction, the watershed transform) is built by combining them. Morphology is the standard toolbox for cleaning binary masks, measuring shapes, and post-processing segmentations.

The field was founded in 1964 by Georges Matheron and Jean Serra at the École des Mines de Paris, in work on the quantitative petrography of iron ore from Lorraine: Serra’s 1964 report introduced the structuring element and the hit-or-miss transformation, and Matheron’s study of granulometries followed; the school created its Centre de Morphologie Mathématique in 1968 (Matheron & Serra, 2002). Serra’s book Image Analysis and Mathematical Morphology (1982) became a standard reference. The tutorial by Haralick, Sternberg, and Zhuang (1987) reviewed binary and grayscale morphology, arguing that for tasks such as object identification and defect inspection, morphological operators relate more directly to shape than convolution does.

Definition

Morphology treats a binary image as a set (its foreground pixels) and a grayscale image as a function, transformed with a structuring element: a small set of pixel offsets with a designated origin, such as a 3 × 3 square, a disk, or a line segment. Erosion keeps the points where the structuring element fits entirely inside the set; dilation keeps the points where it touches the set. More abstractly, the operators are defined on complete lattices (ordered sets in which every collection has a greatest lower and least upper bound), which covers binary, grayscale, and other ordered data with one theory.

Intuition

Picture the structuring element as a small stamp sliding over the image. Erosion asks, at each position, whether the whole stamp lies inside the foreground; objects shrink, thin parts and small specks vanish, and holes grow. Dilation asks whether the stamp overlaps the foreground at all; objects grow, small holes and gaps fill, and nearby objects merge.

Erosion followed by dilation, the opening, removes everything the stamp cannot fit inside (specks, thin protrusions, narrow bridges) and then restores the surviving objects to roughly their original size. Dilation followed by erosion, the closing, fills everything the stamp cannot fit into from outside (small holes, cracks, narrow gaps).

Formal Definition

Let $A \subseteq \mathbb{Z}^2$ be the foreground of a binary image and $B$ the structuring element. Write $B_{\mathbf{x}} = \{\mathbf{b} + \mathbf{x} : \mathbf{b} \in B\}$ for $B$ translated to $\mathbf{x}$, $\hat{B} = \{-\mathbf{b} : \mathbf{b} \in B\}$ for its reflection, and $A^c$ for the complement (background). Erosion and dilation are

$$A \ominus B = \{\mathbf{x} : B_{\mathbf{x}} \subseteq A\}, \qquad A \oplus B = \{\mathbf{a} + \mathbf{b} : \mathbf{a} \in A,\ \mathbf{b} \in B\} = \{\mathbf{x} : \hat{B}_{\mathbf{x}} \cap A \neq \emptyset\}.$$

Dilation is Minkowski addition of the two sets. Opening and closing are the compositions

$$A \circ B = (A \ominus B) \oplus B, \qquad A \bullet B = (A \oplus B) \ominus B.$$

The opening is the union of all translates of $B$ that fit inside $A$; the closing is the complement of the union of all translates of $\hat{B}$ that fit inside the background.

Grayscale morphology

For a grayscale image $f$ and a structuring function $b$ defined on a domain $B$, dilation and erosion replace union and intersection by maximum and minimum:

$$(f \oplus b)(\mathbf{x}) = \max_{\mathbf{z} \in B} \big[ f(\mathbf{x} - \mathbf{z}) + b(\mathbf{z}) \big], \qquad (f \ominus b)(\mathbf{x}) = \min_{\mathbf{z} \in B} \big[ f(\mathbf{x} + \mathbf{z}) - b(\mathbf{z}) \big].$$

With a flat structuring element ($b = 0$ on $B$), the most common choice, dilation is a moving maximum and erosion a moving minimum over the window $B$ (reflected for dilation). On a 0/1 image they reduce to the set definitions, and opening and closing are the same compositions. Grayscale morphology can also be derived from the binary case by treating the region under the graph of $f$ (its umbra) as a set.

The flat operators are rank filters: the minimum, the median, and the maximum of a window are its lowest, middle, and highest order statistics. The median filter is therefore a close relative of erosion and dilation: it is increasing, and the median of $2k+1$ values equals the maximum, over all subsets of $k+1$ values, of the subset minimum.

Properties

• Increasing. All four operators preserve inclusion: if $A \subseteq A'$, then $A \ominus B \subseteq A' \ominus B$, and likewise for dilation, opening, and closing.
• Extensive and anti-extensive. The opening is anti-extensive ($A \circ B \subseteq A$) and the closing extensive ($A \subseteq A \bullet B$) for any structuring element. If $B$ contains its origin, erosion is anti-extensive and dilation extensive, giving the chain $A \ominus B \subseteq A \circ B \subseteq A \subseteq A \bullet B \subseteq A \oplus B$.
• Idempotence. $(A \circ B) \circ B = A \circ B$ and $(A \bullet B) \bullet B = A \bullet B$. A second opening with the same element changes nothing, which is why openings and closings are called morphological filters.
• Duality. Erosion and dilation are dual with respect to complementation, $(A \ominus B)^c = A^c \oplus \hat{B}$, and so are opening and closing, $(A \circ B)^c = A^c \bullet \hat{B}$.
• Adjunction. $A \oplus B \subseteq C$ holds exactly when $A \subseteq C \ominus B$; this pairing is what makes opening and closing idempotent.
• Composition. Dilation is commutative and associative, so dilating by $B_1$ and then $B_2$ equals dilating by $B_1 \oplus B_2$, and eroding by $B_1$ then $B_2$ equals eroding by $B_1 \oplus B_2$. A large square is thus a horizontal segment followed by a vertical one, which makes large operators cheap.
• Not linear and not invertible. Dilation distributes over unions (pointwise maxima) and erosion over intersections (minima), but neither distributes over sums, and an erosion that deletes a speck cannot be undone.

Computed directly, each operator costs $O(|B|)$ comparisons per pixel; decompositions reduce this.

Examples

Cleaning a noisy binary mask

The example corrupts a binary shape (a disk and a 3-pixel-wide bar) with isolated foreground specks and small holes, cleans it with an opening followed by a closing using a 3 × 3 square, and checks the inclusion chain, idempotence, and duality numerically. Images are padded before each operation so that the image border does not act as background for one operator and as foreground for its dual.

import numpy as np
from scipy import ndimage as ndi

rng = np.random.default_rng(0)

# A binary shape: a filled disk and a 3-pixel-wide bar, corrupted by
# isolated foreground specks outside and small holes inside.
h = w = 128
yy, xx = np.mgrid[0:h, 0:w]
clean = ((yy - 64) ** 2 + (xx - 50) ** 2 <= 30 ** 2) | ((abs(yy - 64) <= 1) & (xx > 80) & (xx < 120))
noise = rng.random((h, w))
img = clean.copy()
img[noise < 0.02] = True            # specks
img[noise > 0.98] = False           # holes

B = np.ones((3, 3), bool)           # 3x3 square structuring element (symmetric)

def apply(op, a, **kw):
    # Pad with background so the image border does not cut into the result.
    return op(np.pad(a, 4), B, **kw)[4:-4, 4:-4]

ero = apply(ndi.binary_erosion, img)
dil = apply(ndi.binary_dilation, img)
opn = apply(ndi.binary_opening, img)
cls = apply(ndi.binary_closing, img)
denoised = apply(ndi.binary_closing, opn)          # open, then close

n = lambda a: int(a.sum())
print("foreground pixels  clean", n(clean), "| noisy", n(img), "| erosion", n(ero),
      "| dilation", n(dil), "| opening", n(opn), "| closing", n(cls))
print("connected components  noisy", ndi.label(img)[1], "| open-then-close", ndi.label(denoised)[1])
print("pixels differing from clean  noisy", n(img ^ clean), "| open-then-close", n(denoised ^ clean))

# Ordering: erosion <= opening <= image <= closing <= dilation (as sets)
sub = lambda a, b: bool(np.all(a <= b))
print("erosion ⊆ opening ⊆ image ⊆ closing ⊆ dilation:",
      sub(ero, opn) and sub(opn, img) and sub(img, cls) and sub(cls, dil))

# Idempotence: opening an opened image changes nothing (same for closing)
print("opening idempotent:", np.array_equal(apply(ndi.binary_opening, opn), opn))
print("closing idempotent:", np.array_equal(apply(ndi.binary_closing, cls), cls))

# Duality via complements (B is symmetric, so its reflection is itself).
# The complement's border is foreground, so pad it with foreground.
comp = lambda op, a: ~op(np.pad(~a, 4, constant_values=True), B, border_value=1)[4:-4, 4:-4]
print("dilation == complement of eroded complement:", np.array_equal(dil, comp(ndi.binary_erosion, img)))
print("opening  == complement of closed complement:", np.array_equal(opn, comp(ndi.binary_closing, img)))

# Grayscale morphology with a flat 3x3 element is a 3x3 min / max filter.
f = rng.integers(0, 256, (64, 64)).astype(np.uint8)
print("grey erosion == 3x3 minimum filter:",
      np.array_equal(ndi.grey_erosion(f, size=(3, 3)), ndi.minimum_filter(f, size=3)))
print("grey dilation == 3x3 maximum filter:",
      np.array_equal(ndi.grey_dilation(f, size=(3, 3)), ndi.maximum_filter(f, size=3)))

Output:

foreground pixels  clean 2938 | noisy 3144 | erosion 2175 | dilation 5385 | opening 2841 | closing 3269
connected components  noisy 250 | open-then-close 1
pixels differing from clean  noisy 320 | open-then-close 29
erosion ⊆ opening ⊆ image ⊆ closing ⊆ dilation: True
opening idempotent: True
closing idempotent: True
dilation == complement of eroded complement: True
opening  == complement of closed complement: True
grey erosion == 3x3 minimum filter: True
grey dilation == 3x3 maximum filter: True

The noise splits the foreground into 250 connected components; after opening and closing only the shape remains, as a single component. The bar survives because the 3 × 3 square fits inside it; a 5 × 5 square would delete it with the specks. The 29 remaining differences all lie on the disk’s boundary, which the filters reshape to suit the square: morphological filters impose the geometry of the structuring element on the result.

Derived operators

• Morphological gradient. The difference between dilation and erosion, $(f \oplus B) - (f \ominus B)$, is large where intensity changes within the window and outlines object boundaries. It is a nonlinear alternative to derivative-based image gradients for edge detection; the half-gradients, dilation minus image and image minus erosion, give outer and inner boundaries.
• Top-hat transforms. The white top-hat $f - (f \circ B)$ keeps bright features smaller than $B$ and removes everything the opening preserves, including a slowly varying background; the black top-hat $(f \bullet B) - f$ does the same for dark features.
• Hit-or-miss transform. With two disjoint structuring elements, $B_1$ for required foreground and $B_2$ for required background, $(A \ominus B_1) \cap (A^c \ominus B_2)$ marks the positions where a specific local pattern occurs, such as an isolated pixel, a line end, or a corner.
• Skeletons and thinning. A skeleton reduces a shape to a thin set of curves along its medial axis that preserves its topology. Morphological skeletons can be computed as a union over sizes $n$ of the residues between the erosion by $nB$ and its opening; thinning repeatedly removes boundary pixels identified by hit-or-miss patterns until only a one-pixel-wide skeleton remains.
• Granulometry. Opening with structuring elements of increasing size removes ever larger structures; the amount of foreground removed at each size forms a size distribution, used to characterize particles and textures.
• Reconstruction. Morphological reconstruction repeatedly dilates a marker image while constraining it to stay beneath a mask image, until nothing changes. In the binary case it recovers exactly the connected components of the mask that contain a marker. Opening by reconstruction erodes to delete small objects, then reconstructs the survivors under the original, so they keep their exact shapes; reconstruction from the image border fills holes or removes border-touching objects. Vincent (1993) described grayscale reconstruction, efficient algorithms for computing it, and its applications to image filtering and segmentation.
• Watershed. The watershed transform treats a grayscale image, typically a gradient magnitude, as a topographic surface and divides it into catchment basins, one per regional minimum. Combined with markers obtained by reconstruction, it is a classical morphological segmentation method.

Choosing the structuring element

The structuring element encodes what “small” and “thin” mean. Its size should sit between the size of the structures to remove and the size of the structures to keep. Its shape sets the geometry of the result: a square produces axis-aligned corners, a disk gives nearly isotropic results, and line segments select oriented structures such as scratches or rows of text. The origin matters for asymmetric elements, and connectivity (4- or 8-neighborhood) matters for reconstruction and skeletonization.

Common Misconceptions

• “Opening and closing undo each other.” Neither inverts the other; open-then-close and close-then-open are different filters, and both lose information.
• “Erosion followed by dilation restores the original.” That is the opening; only structures that are unions of translates of $B$ survive intact.
• “The median filter is a morphological filter.” It is increasing, but not idempotent, so it is not a morphological filter in the technical sense of an opening or closing.

Where It Is Used

• Binary mask clean-up. Opening and closing remove specks and fill holes in masks produced by thresholding (see image histogram), background subtraction, or segmentation networks, before connected-component labeling counts and measures objects.
• Segmentation post-processing. Hole filling, removal of small or border-touching components, separation of touching objects with watersheds, and boundary smoothing.
• Document analysis. Removing noise from scanned pages, joining broken characters, and finding text lines and table rulings with line elements.
• Biomedical and industrial inspection. Top-hats extract small bright or dark spots, such as cells, nuclei, or defects, against an uneven background; granulometries measure particle size distributions.
• Grayscale noise removal. Alternating openings and closings remove impulse-like noise, an alternative to median filtering.
• Deep learning. Max pooling is a flat grayscale dilation followed by subsampling. Researchers have also built network layers from erosion and dilation with learnable structuring elements; Franchi, Fehri, and Yao (2020), for example, proposed morphological layers that can be trained end-to-end inside deep networks and reported boundary detection results with a small fraction of the parameters of competing methods.