Mean Squared Error

The mean squared error (MSE) is the average of the squared differences between a prediction and its reference: between a restored image and the clean original, a predicted depth map and the measured one, or regressed coordinates and annotated ones. Its square root, the root mean squared error (RMSE), is in the units of the data, and the mean absolute error (MAE) averages the absolute differences instead. All three are zero for a perfect prediction and grow with the size of the errors, but they weigh large and small errors differently, and none of them depends on where in the image the errors are. The squared and absolute errors as training losses are covered in Loss Function; this article treats them as evaluation metrics.

Formula

For a reference image $x$ and a prediction $\hat{x}$ with $N$ pixels, write the error at pixel $i$ as $e_i = \hat{x}_i - x_i$. Then

$$\mathrm{MSE} = \frac{1}{N} \sum_{i=1}^{N} e_i^2, \qquad \mathrm{RMSE} = \sqrt{\mathrm{MSE}}, \qquad \mathrm{MAE} = \frac{1}{N} \sum_{i=1}^{N} \lvert e_i \rvert .$$

Wang and Bovik (2009) write the MSE in this form for two signals of $N$ samples and note that the Euclidean ($\ell_2$) distance between them, here $\sqrt{N \cdot \mathrm{MSE}}$, is proportional to the square root of the MSE; MAE is likewise the $\ell_1$ distance divided by $N$.

Channels. For a colour image of $H \times W$ pixels and $C$ channels, the usual definition averages over all $N = HWC$ values. Because every channel has the same number of values, this equals the mean of the per-channel MSEs (an algebraic identity). Summing over channels instead of averaging multiplies the result by $C$, and some papers compute the error on a single channel, such as luma; the digital image article covers the representations, and Peak Signal-to-Noise Ratio covers the channel conventions of restoration benchmarks.

Averaging over a dataset. For a test set of $M$ images, two conventions give different numbers, and papers do not always say which they used (the following is our own derivation):

• Pooled over all pixels: $\mathrm{MSE}_{\text{pool}} = \sum_m \sum_i e_{m,i}^2 \,/\, \sum_m N_m$. Each pixel counts equally, so larger images count more.
• Per image, then averaged: $\overline{\mathrm{MSE}} = \frac{1}{M}\sum_m \mathrm{MSE}_m$. Each image counts equally.

For MSE and MAE the two agree whenever all images have the same size. For RMSE they do not agree even then: because the square root is concave, for equal-sized images the mean of the per-image RMSEs is at most the pooled RMSE, with equality only when every image has the same MSE (Jensen’s inequality). With two equal-sized images, one perfect and one with MSE 400, the mean of per-image RMSEs is $(0 + 20)/2 = 10$, but the pooled RMSE is $\sqrt{200} \approx 14.1$. The same distinction applies to any metric computed from the MSE by a nonlinear function, which includes PSNR. Pixels without ground truth, such as missing depth readings or occluded flow, are usually excluded from both sums, so the number of valid pixels, not the image size, sets the weights.

Intuition

Squares punish large errors. One pixel off by 100 contributes as much to the MSE as 100 pixels off by 10; in the MAE the hundred small errors count ten times as much as the single large one. RMSE is therefore dominated by the largest errors, MAE by the typical one, and the ratio between them says something about the error distribution: in general $\mathrm{MAE} \le \mathrm{RMSE} \le \max_i \lvert e_i \rvert$, with $\mathrm{MAE} = \mathrm{RMSE}$ exactly when every $\lvert e_i \rvert$ is the same, and for zero-mean Gaussian errors $\mathrm{MAE}/\mathrm{RMSE} \to \sqrt{2/\pi} \approx 0.80$ (standard results, from the Cauchy–Schwarz inequality and the mean of a half-normal distribution).

What each metric rewards. With a flexible enough model, minimizing the expected squared error gives the conditional mean of the target and minimizing the expected absolute error gives the conditional median (Goodfellow, Bengio & Courville, 2016, §6.2.1.2; see Loss Function). Evaluating with MSE therefore favours predictors that output conditional means, and with MAE those that output conditional medians. When the target is ambiguous, for example the texture behind a blurred patch, the conditional mean is a blend of the possibilities, which is one reason MSE-optimal restorations look smooth.

A likelihood in disguise. Under independent Gaussian noise of fixed variance, a lower MSE means a higher likelihood (Goodfellow et al., 2016, §5.5.1, eqs. 5.64–5.66); the derivation, and the Laplace-noise counterpart for the absolute error, are in Loss Function and Least Squares. As a metric, MSE is the natural score when the errors really are Gaussian and independent across pixels, as in the additive white Gaussian noise setting of image noise models, and a poorer one when they are not.

Bias and variance. For an estimator $\hat{\theta}$ of a quantity $\theta$, the expected squared error splits into two parts (Goodfellow et al., 2016, §5.4.4, eq. 5.54):

$$\mathbb{E}\big[(\hat{\theta} - \theta)^2\big] = \mathrm{Bias}(\hat{\theta})^2 + \mathrm{Var}(\hat{\theta}).$$

A low MSE needs both a small systematic error and a small spread. The same algebra applies to the errors across one image: $\mathrm{MSE} = \bar{e}^2 + \frac{1}{N}\sum_i (e_i - \bar{e})^2$, the squared mean error plus the variance of the errors. A constant brightness offset is all bias; zero-mean noise is all variance. Reporting the mean error alongside the MSE separates the two.

Interpretation

MSE ranges from 0 (identical) upward without a fixed bound; for images it is at most $L^2$, where $L$ is the width of the intensity range, since no error can exceed $L$. Its units are the squared units of the data, so RMSE and MAE are easier to read: an RMSE of 12 on an 8-bit image means errors of about 12 grey levels in the root-mean-square sense.

The numbers depend on the intensity scale. The same pair of images stored in $[0, 255]$ and in $[0, 1]$ have MSEs that differ by a factor of $255^2 = 65{,}025$, and RMSE and MAE that differ by a factor of 255. A reported MSE of 0.0023 and one of 152 can describe identical results, so an MSE without its range is not interpretable. Depth and coordinates have physical units: depth RMSE is in the units of the depth maps, usually metres, and 3D keypoint errors in millimetres. Clipping and rounding the prediction to valid 8-bit values before scoring also changes the result slightly, so evaluation code should state whether it does.

There is no universal threshold for a good MSE. What counts as good depends on the content (textured images are harder to restore than flat ones), the range, and the task, so MSE is mainly used to compare methods on the same data under the same protocol. Wang and Bovik (2009) list the properties that make it convenient: it needs no parameters, its square root is proportional to the Euclidean distance, a valid metric, it is preserved by orthogonal transforms such as the orthonormal DCT or Fourier transform (Parseval’s theorem), so errors can be measured in a transform domain, and the MSEs of independent zero-mean distortions applied in sequence add, in expectation.

When to Use It

MSE, RMSE and MAE fit any output that is a set of real numbers with a reference of the same shape:

• Restoration and reconstruction. Image denoising, super-resolution, inpainting and compression compare outputs with the clean image, usually through PSNR, a logarithmic rescaling of the MSE (see Peak Signal-to-Noise Ratio); Wang and Bovik (2009) note that competing signal processing algorithms have most often been compared by MSE or PSNR. Rendered views in novel view synthesis are scored the same way.
• Depth. Depth estimation papers report the RMSE of depth and of log depth alongside relative errors and threshold accuracies, the set of error measures from earlier work that Eigen, Puhrsch and Fergus (2014) evaluated with; the depth estimation error metrics article covers those conventions.
• Geometric regression. Many vision metrics average a per-element Euclidean distance rather than a squared one. Endpoint error for optical flow averages $\lVert \hat{\mathbf{u}}_i - \mathbf{u}_i \rVert_2$ over pixels, and the mean per-joint position error of Human3.6M averages the Euclidean distance between predicted and true joints, in millimetres in 3D or pixels in 2D, then averages over frames (Ionescu et al., 2014). These are means of $\ell_2$ norms, like an MAE over vectors: a single large error raises them in proportion, not quadratically as in an RMSE.

Report RMSE when large errors matter more than small ones, MAE when every unit of error costs the same or when the data contain a few corrupt values, and both when the reader needs to see how heavy the tail of the errors is.

Failure Cases

Outliers. A few wildly wrong values can dominate the MSE: in the example below, salt-and-pepper corruption of under 1% of the pixels reaches the same MSE as Gaussian noise on every pixel. An MSE comparison then ranks methods by how they handle the outliers, and robust statistics such as the MAE, a median error or the outlier percentages of Endpoint Error tell a different story.

Errors in different places count the same. Wang and Bovik (2009) name four implicit assumptions of the MSE and of every $\ell_p$ distance: that fidelity does not depend on the spatial arrangement of the samples (reordering both images the same way leaves the MSE unchanged), on any relationship between the error and the image, on the signs of the errors, or on which samples are affected. Their Figure 3 shows a counterexample to each; for instance, adding a constant to every pixel and adding the same constant with random signs give the same MSE but look nothing alike.

Equal MSE, different images. Their Figure 2 alters an image of Einstein with a contrast stretch, a luminance shift, additive Gaussian noise, impulse noise, JPEG compression and blur, all with MSE between 306 and 313, yet with visibly very different quality. A small zoom, shift or rotation of the same image, which the authors describe as a negligible loss of perceived quality, gives MSEs of 577 to 873, roughly two to three times as large. The structural similarity index (SSIM), which compares local means, variances and correlations, scores the six equal-MSE images between 0.576 and 0.987 in the same figure.

Perceptual quality. Because of these assumptions, MSE agrees only loosely with human judgments of image quality, and image quality assessment developed metrics that model what viewers notice: SSIM and learned metrics such as LPIPS. MSE also shapes what methods produce. Restoration networks trained or selected on MSE favour the smooth conditional mean, and Loss Function describes a case where a network trained on absolute error reached a lower squared error than one trained on squared error.

Misalignment. Any per-pixel metric penalizes a prediction that is correct but slightly displaced. A one-pixel offset produces large errors on edges, thin structures and fine texture and almost none in flat regions, so the MSE of a misregistered result depends mostly on how much detail the image contains.

Example

Six distortions of one synthetic grayscale image, with intensities in $[0, 255]$, are tuned to the MSE of a one-pixel horizontal shift: additive Gaussian noise, a constant brightness increase, a Gaussian blur, JPEG-like compression (8×8 block DCT quantized with the luminance table of ITU-T T.81, Annex K), salt-and-pepper impulses, and the shift itself. Besides MSE, RMSE and MAE, the code prints the largest error, the mean error, a neighbour statistic (the average product of horizontally adjacent errors divided by the MSE: 1 for a constant error, 0 for independent errors), and the share of the squared error that falls on the 10% of pixels with the strongest gradient.

import numpy as np
from scipy.fft import dctn, idctn
from scipy.ndimage import gaussian_filter
from scipy.optimize import brentq

rng = np.random.default_rng(0)

# A 128x128 grayscale test image with intensities in [0, 255] (kept as floats):
# a horizontal ramp, a bright disk, a dark square and a patch of fine stripes.
yy, xx = np.mgrid[0:128, 0:128].astype(float)
ref = 60 + 130 * xx / 127
ref[(yy - 40) ** 2 + (xx - 40) ** 2 < 22 ** 2] = 215
ref[80:115, 15:50] = 45
ref[70:120, 70:120] += 30 * np.sin(2 * np.pi * xx[70:120, 70:120] / 6)

def mse(a, b):
    return np.mean((a - b) ** 2)

# One-pixel shift to the right (the first column is repeated). Its MSE sets
# the target that every other distortion is tuned to.
shifted = ref.copy()
shifted[:, 1:] = ref[:, :-1]
target = mse(ref, shifted)

# The JPEG luminance quantization table (ITU-T T.81, Annex K, Table K.1).
Q = np.array([[16, 11, 10, 16, 24, 40, 51, 61], [12, 12, 14, 19, 26, 58, 60, 55],
              [14, 13, 16, 24, 40, 57, 69, 56], [14, 17, 22, 29, 51, 87, 80, 62],
              [18, 22, 37, 56, 68, 109, 103, 77], [24, 35, 55, 64, 81, 104, 113, 92],
              [49, 64, 78, 87, 103, 121, 120, 101], [72, 92, 95, 98, 112, 100, 103, 99]])

def jpeg_like(img, scale):
    """8x8 blockwise DCT, quantization with steps scale * Q, inverse DCT."""
    out = np.empty_like(img)
    for i in range(0, 128, 8):
        for j in range(0, 128, 8):
            c = dctn(img[i:i + 8, j:j + 8] - 128, norm="ortho")
            step = scale * Q
            out[i:i + 8, j:j + 8] = idctn(step * np.round(c / step), norm="ortho") + 128
    return out

noise = rng.standard_normal(ref.shape)
impulse_order = rng.permutation(ref.size)           # pixels in random order
impulse_val = rng.integers(0, 2, ref.size) * 255.0  # black or white

def impulses(frac):
    out = ref.copy().ravel()
    k = int(round(frac * ref.size))
    out[impulse_order[:k]] = impulse_val[:k]
    return out.reshape(ref.shape)

dist = {
    "Gaussian noise": ref + np.sqrt(target) * noise / noise.std(),
    "brightness shift": ref + np.sqrt(target),
    "Gaussian blur": gaussian_filter(
        ref, brentq(lambda s: mse(ref, gaussian_filter(ref, s)) - target, 0.1, 5)),
    "blockwise DCT": jpeg_like(
        ref, brentq(lambda q: mse(ref, jpeg_like(ref, q)) - target, 0.1, 20, xtol=1e-9)),
    "salt and pepper": impulses(
        brentq(lambda f: mse(ref, impulses(f)) - target, 1e-4, 0.1, xtol=1e-9)),
    "one-pixel shift": shifted,
}

# How the error is structured. nbr: average product of horizontally adjacent
# errors over the mean squared error (1 for a constant error, 0 for white noise).
# edge: share of the squared error on edge pixels (top 10% of gradient magnitude).
gy, gx = np.gradient(ref)
edges = np.hypot(gx, gy) > np.quantile(np.hypot(gx, gy), 0.9)

print(f"{'distortion':17s} {'MSE':>6s} {'RMSE':>5s} {'MAE':>5s} {'max|e|':>6s} "
      f"{'mean e':>6s} {'nbr':>5s} {'edge':>5s}")
for name, d in dist.items():
    e = d - ref
    nbr = np.mean(e[:, :-1] * e[:, 1:]) / np.mean(e ** 2)
    print(f"{name:17s} {mse(ref, d):6.1f} {np.sqrt(mse(ref, d)):5.2f} "
          f"{np.mean(np.abs(e)):5.2f} {np.abs(e).max():6.1f} {e.mean():6.2f} "
          f"{nbr:5.2f} {np.sum(e[edges] ** 2) / np.sum(e ** 2):5.2f}")
print(f"salt and pepper changes {np.mean(dist['salt and pepper'] != ref):.2%} of the pixels")

# The same images with intensities in [0, 1].
r1, d1 = ref / 255, dist["Gaussian noise"] / 255
print(f"\nGaussian noise in [0, 1]: MSE {mse(r1, d1):.6f}, RMSE {np.sqrt(mse(r1, d1)):.4f}, "
      f"MAE {np.mean(np.abs(d1 - r1)):.4f}")
print(f"MSE ratio [0, 255] / [0, 1]: {mse(ref, dist['Gaussian noise']) / mse(r1, d1):.1f}")

Output (NumPy 2.3.5, SciPy 1.16.3):

distortion           MSE  RMSE   MAE max|e| mean e   nbr  edge
Gaussian noise     152.3 12.34  9.87   49.8   0.07  0.00  0.10
brightness shift   152.3 12.34 12.34   12.3  12.34  1.00  0.09
Gaussian blur      152.3 12.34  5.76   69.8  -0.00  0.56  0.36
blockwise DCT      152.3 12.34  8.57  132.3  -0.22  0.56  0.33
salt and pepper    153.3 12.38  1.08  215.0  -0.04  0.00  0.07
one-pixel shift    152.3 12.34  4.23  136.6  -1.02  0.22  0.80
salt and pepper changes 0.88% of the pixels

Gaussian noise in [0, 1]: MSE 0.002342, RMSE 0.0484, MAE 0.0387
MSE ratio [0, 255] / [0, 1]: 65025.0

MSE cannot tell these six images apart (salt and pepper misses the target slightly because it can only corrupt whole pixels), while the MAE spreads them over more than a factor of ten, and the other columns show why.

• The brightness shift is the case $\mathrm{MAE} = \mathrm{RMSE}$: every error has the same size and sign, so the whole MSE is squared bias, the neighbour statistic is 1, and the error ignores edges. Visually it is only a uniform brightening.
• Gaussian noise has zero mean, no correlation between neighbours and errors spread evenly (the edge share is what chance gives); its MAE/RMSE ratio is the Gaussian value $\sqrt{2/\pi}$.
• Salt and pepper concentrates the same squared error in fewer than one pixel in a hundred, so its MAE is a small fraction of its RMSE. Most pixels are exact, which a median filter would exploit.
• The blur and the blockwise DCT produce spatially correlated errors that sit on structure, where blur softens edges and stripes and quantization removes high frequencies and leaves ringing and block boundaries.
• The one-pixel shift concentrates its error most strongly on edges, as thin lines along the disk, the square and the stripes; elsewhere the error is small, and the ramp contributes the small negative mean error.

The last two lines show the scale dependence: rescaling both images changes every number by a fixed factor without changing any comparison between methods.

Variants

• Weighted and masked MSE. A per-pixel weight $w_i$ gives $\sum_i w_i e_i^2 / \sum_i w_i$; a binary mask restricts the average to valid pixels or a region of interest, as depth and flow benchmarks do for pixels with ground truth.
• Log-domain RMSE. Computing the RMSE of $\log \hat{x} - \log x$ measures relative rather than absolute error; Eigen et al. (2014) report it for depth, together with a scale-invariant version.
• Sum of squared errors (SSE) is $N \cdot \mathrm{MSE}$, used in matching costs and in least squares, where the number of samples is fixed.
• Normalized errors divide the MSE by the variance or energy of the reference, so that images with more contrast are not penalized for it; the exact normalization varies by field.

Related Metrics

• PSNR is $10 \log_{10}(L^2/\mathrm{MSE})$ in decibels; it carries the same information as the MSE for a fixed $L$, and Peak Signal-to-Noise Ratio covers its conventions.
• SSIM compares local structure rather than pixel differences, and responds to the error structure that MSE ignores. LPIPS compares deep-network features.
• Endpoint Error and the depth estimation error metrics apply these averages to flow vectors and depth maps.
• Fréchet Inception Distance compares distributions of images rather than pairs, for generative models that have no per-image reference.