Image Histogram

An image histogram counts how many pixels take each intensity value. It throws away where the pixels are and keeps only how often each value occurs, which turns a digital image into a compact summary of its brightness and contrast. Histograms reveal exposure problems at a glance, drive automatic thresholding and contrast enhancement, and, computed over colors or gradient orientations, serve as descriptors for recognition and tracking.

Definition

For a grayscale image with $L$ possible intensity levels (256 for 8-bit images), the histogram assigns to each level the number of pixels that have it. Dividing by the total number of pixels gives the normalized histogram, the fraction of pixels at each level. It is the empirical probability distribution of intensity: the probability that a pixel chosen at random has a given value. Summing it up to each level gives the cumulative histogram, the empirical cumulative distribution function (CDF): the fraction of pixels at or below that level.

For continuous or high-bit-depth data, intensities are grouped into bins, intervals of values that are counted together.

Intuition

Imagine sorting the pixels into 256 buckets by brightness, lined up from black to white. A dark photograph fills the left buckets; a hazy, low-contrast scene fills a narrow group in the middle; a dark object on a light background fills two separate groups. A tall spike in the last bucket means many pixels hit the sensor’s maximum.

Shuffling the pixels of a photograph into random positions leaves its histogram unchanged: a histogram describes the distribution of values, not the image.

Formal Definition

Let $I$ be an image with $N$ pixels and levels $k = 0, \dots, L-1$. The histogram, normalized histogram, and cumulative histogram are

$$h(k) = \#\{\mathbf{x} : I(\mathbf{x}) = k\}, \qquad p(k) = \frac{h(k)}{N}, \qquad c(k) = \sum_{j=0}^{k} p(j).$$

$c$ is non-decreasing, with $c(L-1) = 1$. The $q$-th quantile is the smallest level $k$ with $c(k) \ge q$; the median is the level where $c$ first reaches $0.5$. Global statistics can be computed from the histogram alone, for example the mean and variance:

$$\mu = \sum_{k} k\, p(k), \qquad \sigma^2 = \sum_{k} (k - \mu)^2\, p(k).$$

With bins of width $\Delta$, a value $v$ falls in bin $\lfloor v / \Delta \rfloor$.

Color histograms

A color image has several values per pixel. Two kinds of histogram are used:

• Per-channel (marginal) histograms: one histogram for each channel, such as red, green, and blue. They are small ($3L$ bins) but lose the relationship between channels: an image with red and blue pixels can have the same marginal histograms as one with magenta and black pixels.
• Joint histograms: one count per combination of channel values, a 3D array of bins. A full 8-bit RGB joint histogram has $256^3 \approx 16.8$ million bins, far more than a typical image has pixels, so joint histograms use coarse bins. Swain and Ballard (1991), for example, used 2,048 bins on opponent color axes, quantizing the intensity axis more coarsely (8 bins) than the two chromatic axes (16 each) because intensity is more sensitive to lighting.

The choice of color space matters: histograms of hue and saturation, or of chromaticity, are less affected by brightness changes than RGB histograms.

Properties

What histograms reveal

• Exposure. Mass concentrated at low levels indicates underexposure; at high levels, overexposure.
• Contrast. The spread of the histogram, measured by its standard deviation or by the range between low and high percentiles, measures global contrast. A narrow histogram means the image uses few of the available levels.
• Clipping. Spikes at $0$ or $L-1$ mean values were cut off at the limits of the sensor or encoding. Clipped pixels carry no information about how bright or dark the scene really was.
• Dynamic range and quantization. Gaps and combs (empty bins between occupied ones) are typical of an image whose levels were stretched after quantization. Noise widens every peak (see image noise).

Invariances

A global histogram is unchanged by any rearrangement of pixels, so it is unaffected by translating, rotating, or mirroring the image content (up to resampling and boundary effects). Swain and Ballard note that object color histograms also change only slowly with viewing angle, scale, and partial occlusion. The same property is the main limitation: histograms carry no spatial information.

Binning

Fewer, wider bins give smoother and more stable histograms with less detail; more bins resolve finer differences but become sparse and noisy when few pixels are counted. The trade-off matters most for joint color histograms and for histograms of small regions. Bin boundaries are arbitrary, so two nearly equal values can land in different bins; soft assignment, where each value contributes to neighboring bins with weights, reduces this effect.

Examples

Histogram and CDF of a low-contrast image

The code builds a synthetic 8-bit image with a noisy ramp confined to gray levels 90–150 and a small saturated patch, then computes its histogram, CDF, percentiles, and clipping fractions.

import numpy as np

rng = np.random.default_rng(0)

# A synthetic low-contrast 8-bit image: a noisy horizontal ramp squeezed into
# gray levels 90-150, plus a small overexposed patch clipped at 255.
h, w = 256, 256
ramp = np.tile(np.linspace(90, 150, w), (h, 1)) + rng.normal(0, 4, (h, w))
img = np.clip(np.round(ramp), 0, 255).astype(np.uint8)
img[100:140, 100:140] = 255                     # saturated highlight

hist = np.bincount(img.ravel(), minlength=256)  # count per gray level
p = hist / hist.sum()                           # normalized histogram
cdf = np.cumsum(p)                              # cumulative histogram

print("pixels:", hist.sum(), "| occupied levels:", np.count_nonzero(hist))
for q in [0.01, 0.50, 0.95, 0.99]:
    level = np.searchsorted(cdf, q)             # smallest level with CDF >= q
    print(f"percentile {q * 100:>2.0f}: {level}")
print(f"clipped at 0:   {p[0]:.2%}")
print(f"clipped at 255: {p[255]:.2%}")

Output:

pixels: 65536 | occupied levels: 88
percentile  1: 87
percentile 50: 121
percentile 95: 150
percentile 99: 255
clipped at 0:   0.00%
clipped at 255: 2.44%

Only 88 of the 256 levels are used, and the 1st to 95th percentiles span just 87 to 150: the image is low in contrast. The 2.44% of pixels at 255 is exactly the 40 × 40 saturated patch. Because more than 1% of pixels are clipped, the 99th percentile jumps to 255, so a contrast stretch based on the 1st and 99th percentiles would barely change this image, while one based on the 1st and 95th would map 87–150 onto the full range.

Uses

Thresholding. Separating foreground from background with a single intensity threshold works when the histogram has two well-separated modes. Otsu (1979) proposed choosing the threshold automatically by a discriminant criterion: split the histogram into two classes at level $k$, and pick the $k$ that maximizes the between-class variance

$$\sigma_B^2(k) = \omega_0(k)\, \omega_1(k)\, \big(\mu_0(k) - \mu_1(k)\big)^2,$$

where $\omega_0, \omega_1$ are the fractions of pixels in the two classes and $\mu_0, \mu_1$ their mean intensities. Maximizing it is equivalent to minimizing the within-class variance. Otsu showed that the criterion needs only the zeroth- and first-order cumulative moments of the histogram, so all thresholds can be evaluated in one pass, and that it extends to several thresholds.

Contrast enhancement. Linear contrast stretching maps a chosen low and high percentile to the ends of the range. Histogram equalization goes further and maps each level through the CDF itself, scaled to the output range, so that output levels are used approximately uniformly; it brings out detail in crowded parts of the histogram, but the same stretching can make noise in nearly uniform regions more visible. Adaptive variants equalize local regions separately.

Histogram matching. Also called histogram specification, it transforms an image so its histogram approximates a target histogram: each level is mapped to the target level with the same cumulative value, that is, through the source CDF followed by the inverse of the target CDF. It is used to normalize brightness across cameras or acquisitions before comparison.

Recognition and indexing. Swain and Ballard (1991) showed that color histograms of multicolored objects can identify objects in a database. Their histogram intersection sums, over bins, the minimum of the image and model counts, normalized by the model’s pixel count, so pixels in the background that do not match the model do not reduce the score. They also introduced histogram backprojection to locate a known object: each image pixel is replaced by the value of a ratio histogram, the model count divided by the image count for its color bin and capped at 1, which de-emphasizes colors that are common elsewhere in the image. After convolution with a disk about the size of the object, the peak marks its expected location. They noted that histogram intersection is sensitive to lighting changes unless colors are corrected first.

Tracking. Color histograms are a common appearance model in object tracking. Comaniciu, Ramesh, and Meer (2003) represent the target by a color histogram weighted with a spatial kernel that favors pixels near the center, measure similarity with a metric derived from the Bhattacharyya coefficient, and find the target in each new frame with the mean shift procedure, a gradient-based search for the most similar location. They report that the tracker coped with camera motion, partial occlusion, and clutter.

Local histograms and descriptors. Histograms computed over small regions describe local structure. Dalal and Triggs (2005) built the histogram of oriented gradients (HOG) descriptor from grids of local histograms of gradient orientations, weighted by gradient magnitude, with contrast normalization over overlapping blocks; they reported that fine orientation binning with relatively coarse spatial binning worked well for human detection. Local gradient orientation histograms also underlie widely used keypoint descriptors in feature matching.

Common Misconceptions

• “The histogram describes the image.” Images with completely different content can share a histogram; texture and shape are invisible to it.
• “A good image has a flat histogram.” The ideal histogram depends on the scene: a night scene should be mostly dark, and equalizing it can look unnatural.
• “Two modes guarantee good thresholding.” Overlapping classes, uneven illumination, and small objects can all defeat a global threshold even when the histogram looks bimodal; a small object may not produce a visible mode at all.
• “More bins are always more precise.” With limited pixels, fine bins make sparse, noisy histograms. Swain and Ballard estimated that about 200 distinct colors suffice to distinguish a large number of objects.
• “Per-channel histograms capture color.” They lose the correlations between channels; a joint histogram is needed to describe which colors occur.

Where It Is Used

• Image analysis and enhancement: exposure and clipping checks, Otsu thresholding, contrast stretching, histogram equalization, and histogram matching, often in a perceptual color space.
• Recognition and retrieval: color histograms with histogram intersection, and backprojection for localization.
• Tracking: kernel-weighted color histograms in mean shift tracking and other appearance models for object tracking.
• Descriptors: HOG and other descriptors built from local histograms of image gradients.