Lens Distortion

Lens distortion is the systematic way a real lens departs from the pinhole camera model: image points land closer to or farther from the image center than straight-line projection predicts, so straight lines in the scene appear curved, most visibly near the borders of wide-angle images. Nearly every geometric method in computer vision assumes pinhole projection, so distortion is modeled as part of the camera, estimated during calibration alongside the camera parameters, and either removed from the images or built into every projection.

Definition

Lens distortion is a geometric deviation: a smooth, position-dependent displacement of each image point away from where the ideal pinhole projection of the same ray would fall. It changes where a point is imaged, not how sharply. That separates it from other lens imperfections. Blur from defocus or spherical aberration spreads a point into a patch; vignetting darkens the image toward the corners; chromatic aberration images different wavelengths at slightly different positions or focus distances, producing colored fringes. Under distortion, a point stays a point; only its position moves.

Because distortion is a property of the lens, it is an intrinsic property of the camera. It stays fixed while the camera moves, but changes when the zoom setting changes and can change slightly with focus.

Intuition

Most lenses are designed to be rotationally symmetric about the optical axis, so their dominant error is radial: a point is displaced along the line joining it to the distortion center, by an amount that grows with its distance from that center.

• Barrel distortion shrinks magnification toward the edges. Points far from the center are pulled inward, and a straight line that does not pass through the center bows outward, like the staves of a barrel. It is common in wide-angle lenses.
• Pincushion distortion increases magnification toward the edges. Off-center points are pushed outward, and lines bow inward toward the center. It is often seen at the long end of zoom lenses.
• Mustache (complex) distortion mixes the two, for example barrel-like near the center and pincushion-like near the edges, so that lines take a wavy shape. A single radial coefficient cannot represent it; higher-order terms whose sign differs from $k_1$’s are needed.

Lines through the distortion center stay straight, because radial displacement moves their points along the line itself; every other line bends.

Assembly errors add a smaller, asymmetric component. When lens elements are slightly tilted or decentered relative to each other, displacements gain a component perpendicular to the radial direction, called tangential or decentering distortion. Conrady analyzed such decentred lens systems in 1919.

Formal Definition

The Brown–Conrady model

The standard model in photogrammetry and computer vision is usually called the Brown–Conrady model, after Conrady’s analysis of decentring and its reformulation by the photogrammetrist Duane Brown in his 1966 paper “Decentering Distortion of Lenses” in Photogrammetric Engineering. It acts on normalized image coordinates $(x, y) = (X_c/Z_c, Y_c/Z_c)$, the pinhole projection of a camera-frame point before the intrinsic matrix is applied. In the form used by OpenCV, with $r^2 = x^2 + y^2$:

$$\begin{aligned} x_d &= x \,(1 + k_1 r^2 + k_2 r^4 + k_3 r^6) + 2 p_1 x y + p_2 (r^2 + 2x^2), \\ y_d &= y \,(1 + k_1 r^2 + k_2 r^4 + k_3 r^6) + p_1 (r^2 + 2y^2) + 2 p_2 x y. \end{aligned}$$

The distorted normalized point is then mapped to pixels by the intrinsics, $u = f_x x_d + c_x$ and $v = f_y y_d + c_y$. The radial coefficients $k_1, k_2, k_3$ scale each point by a factor that depends only on $r$; the tangential coefficients $p_1, p_2$ model decentering. In this convention the model maps ideal points to distorted ones: $k_1 < 0$, with the radial factor decreasing in $r$, gives barrel distortion, and $k_1 > 0$ gives pincushion. OpenCV stores the coefficients in the order $(k_1, k_2, p_1, p_2, k_3)$, optionally followed by a rational radial denominator $k_4, k_5, k_6$, thin-prism coefficients $s_1, \dots, s_4$, and a sensor tilt. Its documentation notes that radial distortion of a real lens is monotonic and that a non-monotonic estimate should be treated as a calibration failure: such an estimate can fit well near the center and fail at the edges.

The division model

Fitzgibbon (2001) used a different one-parameter approximation, which he called the division model. It maps a distorted point $\mathbf{x}_d$, measured relative to the distortion center, directly to its undistorted position:

$$\mathbf{x} = \frac{\mathbf{x}_d}{1 + \lambda \lVert \mathbf{x}_d \rVert^2}.$$

He stressed that it is not an approximation of the polynomial model but another approximation of the lens’s true distortion curve; in his one experiment, against calibrated ground truth for a laboratory camera, the division model fit with an RMS error of 0.65 pixels and the one-term polynomial with 0.77. To first order $\lambda \approx k_1$, so barrel distortion again has $\lambda < 0$. Its value is that undistortion is a closed-form expression, and that it takes a simple form in homogeneous coordinates. Fitzgibbon used this to estimate the fundamental matrix and the distortion parameter together from point correspondences, as a quadratic eigenvalue problem fast enough to run inside a robust matching loop.

Fisheye models

Fisheye lenses cover fields of view around 180°, which perspective projection cannot represent, since a ray at 90° to the axis would land at infinity. They are better described by mapping the angle $\theta$ between a ray and the optical axis to an image radius. Kannala and Brandt (2006) list the classical designs (stereographic, equidistance, equisolid-angle, and orthogonal projection), note that equidistance, $r = f\theta$, is perhaps the most common, and propose a generic model that covers all of them as well as conventional lenses:

$$r(\theta) = k_1 \theta + k_2 \theta^3 + k_3 \theta^5 + k_4 \theta^7 + k_5 \theta^9.$$

They found these five terms enough and added asymmetric terms for real lenses. OpenCV’s fisheye module uses a four-parameter version with the first coefficient fixed to 1, writing the distorted angle as $\theta_d = \theta(1 + k_1\theta^2 + k_2\theta^4 + k_3\theta^6 + k_4\theta^8)$ with $\theta = \arctan r$, where $r$ is the radius of the normalized pinhole point.

Properties

Estimating distortion

Distortion coefficients are almost always estimated during camera calibration. A known target, usually a planar checkerboard, is imaged in several poses, and the intrinsics, the poses, and the distortion coefficients are refined together by nonlinear least squares on the reprojection error.

Distortion can also be estimated without a target, from the principle that a pinhole camera images every straight line as a straight line. Brown’s plumb-line method (1971) photographed an array of plumb lines and solved for the distortion parameters that straightened their images. Devernay and Faugeras (2001) automated the idea for scenes of structured environments: they extract edges, find candidate lines by polygonal approximation with a large tolerance, and fit the distortion parameters that best transform those edges into straight segments.

Undistortion

Applying the Brown–Conrady model is easy; inverting it is not. Recovering $(x, y)$ from $(x_d, y_d)$ means solving a polynomial system, which has no closed-form solution in general. The usual remedy is fixed-point iteration: start from $\mathbf{x} = \mathbf{x}_d$, then repeatedly subtract the tangential term evaluated at the current estimate and divide by the radial factor. OpenCV’s point undistortion works this way, by default with five iterations. The iteration converges quickly for moderate distortion but can need more steps, or fail, near the edges of strongly distorted images.

Undistorting a whole image avoids the inversion entirely. Using backward warping, each pixel of the output image is converted to normalized coordinates, passed through the forward distortion model, and mapped back to pixels to find where to sample the input. OpenCV precomputes these source coordinates once as lookup maps and resamples each frame with interpolation.

The output image needs a camera matrix of its own, and the choice is a trade-off. Undistorting barrel distortion stretches the edges outward, so either the output keeps the original field of view and shows empty regions at the borders, or it is scaled up and cropped so that every output pixel is valid. OpenCV exposes this choice as a free scaling parameter when computing the new camera matrix: 0 keeps only valid pixels, 1 retains all source pixels.

Effects on vision

Distortion breaks the straight-line assumption at the core of projective methods. Projective geometry maps lines to lines, so homographies, vanishing points, line fitting, and epipolar constraints all hold only after distortion is removed. Fitzgibbon (2001) noted that when uncorrected images are matched, the fundamental-matrix constraint has to be loosened so much that matching suffers. In structure from motion and SLAM, unmodeled distortion is a systematic error rather than noise, so bundle adjustment cannot average it away and instead absorbs it into biased intrinsics, poses, and structure.

Examples

The example applies the Brown–Conrady model with a barrel coefficient $k_1 = -0.25$ to horizontal lines in normalized coordinates, measures how far each distorted line departs from its best-fitting straight line (converted to pixels for a focal length of 800 pixels), and then undistorts a grid by fixed-point iteration, with and without additional $k_2$ and tangential terms.

import numpy as np

def distort(x, y, k1=0.0, k2=0.0, k3=0.0, p1=0.0, p2=0.0):
    """Brown-Conrady model on normalized coordinates (OpenCV convention)."""
    r2 = x**2 + y**2
    radial = 1 + k1 * r2 + k2 * r2**2 + k3 * r2**3
    xd = x * radial + 2 * p1 * x * y + p2 * (r2 + 2 * x**2)
    yd = y * radial + p1 * (r2 + 2 * y**2) + 2 * p2 * x * y
    return xd, yd

def undistort(xd, yd, iters, **c):
    """Invert distort() by fixed-point iteration, starting from the distorted point."""
    x, y = xd.copy(), yd.copy()
    for _ in range(iters):
        r2 = x**2 + y**2
        radial = 1 + c.get("k1", 0) * r2 + c.get("k2", 0) * r2**2 + c.get("k3", 0) * r2**3
        dx = 2 * c.get("p1", 0) * x * y + c.get("p2", 0) * (r2 + 2 * x**2)
        dy = c.get("p1", 0) * (r2 + 2 * y**2) + 2 * c.get("p2", 0) * x * y
        x, y = (xd - dx) / radial, (yd - dy) / radial
    return x, y

def line_deviation(px, py):
    """Largest perpendicular distance from a total-least-squares line fit."""
    pts = np.stack([px, py], axis=1)
    centered = pts - pts.mean(axis=0)
    normal = np.linalg.svd(centered)[2][-1]  # direction of least variance
    return np.abs(centered @ normal).max()

f = 800.0                                  # focal length in pixels, to report pixel units
coeffs = dict(k1=-0.25)                    # barrel: radial factor decreases with r
xs = np.linspace(-0.6, 0.6, 61)            # roughly a 960-pixel-wide field at f = 800

print("Straight horizontal lines after barrel distortion (k1 = -0.25):")
for y0 in [0.0, 0.2, 0.4]:
    xd, yd = distort(xs, np.full_like(xs, y0), **coeffs)
    print(f"  y = {y0:.1f}: max deviation from straight line = {f * line_deviation(xd, yd):6.2f} px")

xd, yd = distort(np.array([0.6]), np.array([0.4]), **coeffs)
print(f"Corner (0.6, 0.4) -> ({xd[0]:.4f}, {yd[0]:.4f}): pulled toward the center")

gx, gy = np.meshgrid(xs, np.linspace(-0.4, 0.4, 41))
for name, c in [("barrel only", coeffs),
                ("with k2 and tangential", dict(k1=-0.25, k2=0.08, p1=0.002, p2=-0.001))]:
    xd, yd = distort(gx, gy, **c)
    print(f"Fixed-point undistortion, {name}:")
    for n in [1, 2, 5, 10, 20]:
        ux, uy = undistort(xd, yd, n, **c)
        err = f * np.hypot(ux - gx, uy - gy).max()
        print(f"  {n:2d} iterations: max error = {err:.2e} px")

Output:

Straight horizontal lines after barrel distortion (k1 = -0.25):
  y = 0.0: max deviation from straight line =   0.00 px
  y = 0.2: max deviation from straight line =   9.44 px
  y = 0.4: max deviation from straight line =  18.88 px
Corner (0.6, 0.4) -> (0.5220, 0.3480): pulled toward the center
Fixed-point undistortion, barrel only:
   1 iterations: max error = 2.02e+01 px
   2 iterations: max error = 5.88e+00 px
   5 iterations: max error = 1.55e-01 px
  10 iterations: max error = 3.70e-04 px
  20 iterations: max error = 2.10e-09 px
Fixed-point undistortion, with k2 and tangential:
   1 iterations: max error = 1.29e+01 px
   2 iterations: max error = 2.61e+00 px
   5 iterations: max error = 2.21e-02 px
  10 iterations: max error = 7.81e-06 px
  20 iterations: max error = 9.86e-13 px

The line through the center stays straight, and the others bend by up to about 19 pixels, more the farther they lie from the center. The corner moves inward by 13%, as barrel distortion requires. The iteration converges linearly: each step shrinks the error by a roughly constant factor. For this strong distortion, five iterations still leave an error of about 0.15 pixels at the corners, which is why the iteration count matters for wide-angle lenses.

Common Misconceptions

• “Distortion makes images blurry.” Distortion moves points; it does not spread them. A heavily distorted image can be perfectly sharp, and undistortion does not sharpen it (the resampling it requires blurs slightly).
• “This lens has zero distortion.” Lenses marketed as distortion-free, or rectilinear, have distortion small enough for their intended use, not zero. For measurement, even a fraction of a pixel matters, so a camera should be calibrated rather than assumed ideal.
• “The coefficients apply to pixel coordinates.” In the OpenCV and Brown–Conrady convention they apply to normalized coordinates, after subtracting the principal point and dividing by the focal length. Applying them to raw pixel coordinates, or reusing coefficients after changing the image resolution without rescaling the intrinsics, gives wrong results.

Where It Is Used

Camera calibration estimates distortion together with the intrinsic and extrinsic camera parameters, and every downstream geometric method depends on it: homography estimation and image stitching, two-view geometry and stereo rectification, structure from motion, visual SLAM, and photogrammetric measurement. Undistortion maps are computed once per camera and applied to every frame with image warping. Feature matching across wide-angle images either undistorts the keypoints first or, as in Fitzgibbon’s method, estimates distortion jointly with the epipolar geometry. Plumb-line calibration builds on edge detection. Fisheye models serve wide-field cameras on vehicles and robots.