Image Noise

Image noise is the random variation in recorded pixel values that is not caused by the scene. Photograph the same static, evenly lit surface twice with the same settings and the two digital images will not be identical: each pixel fluctuates around its expected value. Some of that fluctuation comes from the physics of light itself, some from the sensor and its electronics, and some from later processing.

Noise matters because almost every vision algorithm compares pixel values, with their neighbors or with pixels in another image, and responds to noise as if it were image structure.

Definition

Write the recorded value at pixel $\mathbf{x}$ as $z(\mathbf{x})$ and its expected value, the average over infinitely many exposures of the same scene, as $y(\mathbf{x}) = \mathbb{E}[z(\mathbf{x})]$. The noise is the difference $n(\mathbf{x}) = z(\mathbf{x}) - y(\mathbf{x})$. By construction it has zero mean; what characterizes it is its distribution, how its variance depends on $y$, and whether values at different pixels are correlated.

Temporal noise changes from one exposure to the next. Spatial nonuniformity, often called fixed-pattern noise, is a pixel-to-pixel variation that repeats in every exposure. The EMVA 1288 camera characterization standard points out that the second is not noise in the temporal sense, and prefers the term nonuniformity.

Physical Sources

The following sources are described in sensor models such as those of Healey and Kondepudy (1994) for CCD cameras and the EMVA 1288 standard.

• Photon shot noise. Light arrives as discrete photons, and the number of photoelectrons a pixel collects during an exposure follows a Poisson distribution, whose variance equals its mean. Shot noise is therefore signal-dependent: brighter pixels have larger absolute fluctuations. It is present even in an ideal sensor.
• Dark current. Thermal energy frees electrons in the silicon that are indistinguishable from photoelectrons. Their expected number grows with exposure time and increases strongly with temperature, and because they are also Poisson distributed, they add both an offset and shot noise of their own.
• Read noise. The amplifier and readout electronics add a signal-independent disturbance, modeled as zero-mean Gaussian. It sets the noise floor and dominates shot noise at low signal levels.
• Quantization noise. The analog-to-digital converter rounds to integer levels. With a quantization step $q$, the rounding error is approximately uniform over $[-q/2, q/2]$, with variance $q^2/12$.
• Spatial nonuniformity. The offset of each pixel varies and so does its gain; EMVA 1288 calls these dark signal nonuniformity (DSNU) and photo response nonuniformity (PRNU). DSNU is visible in dark frames; PRNU scales with the signal, so it matters most in bright regions. Both can be largely removed by calibration with dark and uniformly lit reference frames, as Healey and Kondepudy describe.

Independent sources combine by adding variances, a property of sums of independent random variables (see Gaussian distribution).

Formal Definition

Poisson–Gaussian model

Collecting the temporal sources gives the model that EMVA 1288 uses for linear cameras and that Foi and colleagues (2008) formalized for raw sensor data. The raw value is a Poisson-distributed photon count, scaled by a gain, plus independent Gaussian noise for the signal-independent disturbances. Its variance is then an affine function of the expected value:

$$\operatorname{var}[z(\mathbf{x})] = a\, y(\mathbf{x}) + b,$$

where $a$ depends on the gain (the size of one electron in output units) and $b$ collects read noise and other stationary terms. Two parameters describe the noise at every intensity. Foi and colleagues also model clipping at under- and overexposure, which changes the mean–variance relation near the ends of the range, and estimate $a$ and $b$ from a single raw image. Because a Poisson distribution with a large mean is close to a Gaussian with equal mean and variance, the model is often used as a Gaussian with signal-dependent variance $a y + b$.

Additive white Gaussian noise

The most common simplified model, especially in denoising research, is additive white Gaussian noise (AWGN):

$$z(\mathbf{x}) = y(\mathbf{x}) + n(\mathbf{x}), \qquad n(\mathbf{x}) \sim \mathcal{N}(0, \sigma^2) \text{ independently at every pixel}.$$

“White” means uncorrelated across pixels, which gives a flat power spectrum. AWGN is the special case $a = 0$ of the Poisson–Gaussian model. It is reasonable when read noise dominates or over a narrow intensity range, but not for a raw sensor across its whole range.

Impulse noise

Salt-and-pepper or impulse noise replaces a small fraction of pixels with extreme values (minimum or maximum) or random values, leaving the rest untouched. It can arise from defective (hot or dead) pixels and from transmission or storage errors. Its distribution is nothing like a Gaussian: most pixels are exact and a few are wildly wrong, which is why it calls for robust, rank-based methods such as the median filter rather than linear smoothing.

Properties

Signal-to-noise ratio and light

The signal-to-noise ratio (SNR) is the mean signal divided by the noise standard deviation. In electron units, following EMVA 1288, with $N$ collected photoelectrons and read noise $\sigma_r$ (ignoring dark current and quantization),

$$\mathrm{SNR} = \frac{N}{\sqrt{N + \sigma_r^2}}.$$

At high signal, shot noise dominates and $\mathrm{SNR} \approx \sqrt{N}$: collecting four times as much light only doubles the SNR. At low signal, read noise dominates and the SNR grows linearly with $N$. The $\sqrt{N}$ limit is physical: only collecting more light per pixel (larger pixels, wider apertures, longer exposures, or averaging frames) raises it. EMVA 1288 also notes that, apart from a small quantization effect, the camera’s overall gain cancels out of the SNR: amplifying the signal after collection amplifies the noise with it.

Processing changes noise

The models above describe raw sensor data. A camera pipeline then changes the noise at each step:

• Demosaicing interpolates missing color samples from neighbors, so noise at neighboring pixels and in different channels becomes correlated.
• Nonlinear tone curves change how noise depends on intensity: a curve that brightens shadows also amplifies their noise.
• Denoising and sharpening remove some frequencies and boost others, so the residual noise is no longer spatially white.
• Lossy compression quantizes transform coefficients, adding structured errors such as block artifacts.

Noise in a finished JPEG is therefore generally neither white nor of a fixed variance, and methods tuned for white Gaussian noise may perform poorly on it.

Measuring noise

• Dark frames, taken with no light, show offset, DSNU, dark current, and read noise. Averaging many of them gives a per-pixel offset map.
• Flat fields, taken of a uniform, evenly lit target, show PRNU and shot noise. EMVA 1288 prescribes pairs of such images at a series of exposure levels: the variance of their difference isolates temporal noise from fixed patterns, and the straight-line relation between variance and mean, the photon transfer method, gives the camera gain from the slope and the dark noise from the offset.
• Noise level estimation from a single image looks for regions without structure, or uses high-pass components (for example wavelet detail coefficients) that contain mostly noise, and fits a noise model to local mean–variance pairs. Foi and colleagues’ method follows this approach for the Poisson–Gaussian model.

Examples

Shot noise and read noise at several light levels

The code simulates a uniformly lit patch at five light levels: a Poisson number of photoelectrons per pixel, plus Gaussian read noise of 3 electrons. It compares the measured variance with the Poisson–Gaussian prediction $N + \sigma_r^2$, and the measured SNR with the formula above and with the shot-noise limit $\sqrt{N}$.

import numpy as np

rng = np.random.default_rng(0)
read_noise = 3.0          # read-noise standard deviation, in electrons
n_pixels = 100_000        # pixels in a uniformly lit patch

print(f"{'photons':>7} {'variance':>9} {'N + 9':>7} {'SNR':>7} "
      f"{'predicted':>9} {'sqrt(N)':>8}")
for n in [1, 10, 100, 1000, 10000]:
    # Each pixel collects a Poisson number of photoelectrons (quantum
    # efficiency 1), then the readout adds Gaussian noise.
    electrons = rng.poisson(n, n_pixels).astype(float)
    signal = electrons + rng.normal(0.0, read_noise, n_pixels)
    snr = signal.mean() / signal.std()
    predicted = n / np.sqrt(n + read_noise**2)
    print(f"{n:>7} {signal.var():>9.1f} {n + read_noise**2:>7.0f} "
          f"{snr:>7.2f} {predicted:>9.2f} {np.sqrt(n):>8.2f}")

Output:

photons  variance   N + 9     SNR predicted  sqrt(N)
      1      10.0      10    0.32      0.32     1.00
     10      19.0      19    2.29      2.29     3.16
    100     109.3     109    9.57      9.58    10.00
   1000    1009.4    1009   31.48     31.48    31.62
  10000    9993.2   10009  100.03     99.96   100.00

The variance grows linearly with the signal, as the affine model predicts. At 1 and 10 photoelectrons the read noise holds the SNR well below $\sqrt{N}$; from about 1,000 photoelectrons on, the SNR is essentially the shot-noise limit, and each tenfold increase in light raises it only by about $\sqrt{10} \approx 3.2$.

Why noise matters in vision

• Gradients. A finite difference of two pixels with independent noise of variance $\sigma^2$ has noise variance $2\sigma^2$, while the signal difference between neighbors in a smooth region is small. Derivatives amplify noise relative to signal, which is why image gradients are computed after smoothing, usually by convolution with a Gaussian.
• Matching and motion. Patch comparisons in feature matching, stereo, and optical flow rely on brightness constancy, which noise violates at every pixel. Costs are therefore aggregated over windows, and uniform regions, where signal differences are below the noise, cannot be matched reliably.
• Estimation. Sum-of-squared-differences costs and least squares fits are maximum-likelihood estimates under independent Gaussian noise of equal variance. With signal-dependent noise, bright and dark pixels deserve different weights.

Common Misconceptions

• “Image noise is Gaussian with a fixed variance.” Raw sensor noise is signal-dependent (Poisson–Gaussian), and processing makes it spatially correlated. AWGN is a modeling convenience.
• “A better sensor eliminates noise.” Shot noise is a property of light. A sensor can reduce read noise, dark current, and nonuniformity, but at a given number of collected photons the SNR cannot exceed $\sqrt{N}$.
• “Fixed-pattern noise is random noise.” It repeats in every frame, so averaging frames does not remove it; calibration with dark and flat frames does.
• “High ISO causes the noise.” Raising ISO raises the gain, which by the EMVA 1288 model does not by itself change the SNR apart from quantization. High-ISO images are noisy mainly because high ISO is chosen when little light is available, and the shortage of photons limits the SNR.

Where It Is Used

• Filtering and differentiation: smoothing before computing image gradients, and robust filters such as the median filter for impulse noise.
• Denoising: image denoising methods, from Gaussian and bilateral filters to non-local means and learned denoisers, start from an assumed noise model and often an estimated noise level.
• Camera characterization: dark-frame and flat-field calibration, photon transfer measurements, and sensor comparison under EMVA 1288.