Disparity

Disparity is the difference in image position of the same scene point in two views. In a rectified stereo pair it is one number per pixel, the horizontal shift between the left and right images, and it is inversely proportional to depth. Stereo matching outputs disparity, not depth, so how disparity is defined, stored, and converted links matching to 3D measurement.

Definition

For a rectified pair with the left image as reference, a scene point that appears at column $x_L$ in the left image and at $x_R$ on the same row of the right image has disparity $d = x_L - x_R$, measured in pixels. A disparity map assigns such a value to every pixel of the reference image, with some pixels marked as having no value.

Intuition

Seen from two positions side by side, near objects shift a lot and far objects barely move. Disparity is the size of that shift: depth information in image units, so the same 10-pixel shift means a different distance on every rig.

Formal Definition

For two identical rectified cameras with focal length $f$ in pixels and baseline $B$, the stereo vision article derives

$$d = x_L - x_R = \frac{fB}{Z}, \qquad Z = \frac{fB}{d}.$$

Disparity is therefore inverse depth scaled by the rig constant $fB$, which Scharstein and Szeliski (2002) call the classical inverse-depth interpretation.

Sign and direction. With the left image as reference and the right camera displaced to the right, a point in front of the cameras appears further left in the right image, so $x_L > x_R$ and $d > 0$; $d \to 0$ as $Z \to \infty$. The left pixel $(x, y)$ matches the right pixel $(x - d, y)$. A right-reference map is usually stored positive too, matching right pixel $(x, y)$ to left pixel $(x + d, y)$. Scharstein and Szeliski write the correspondence with a sign $s = \pm 1$ chosen so that disparities are always positive; since this is a convention, check it before combining maps from different sources.

Principal-point offset. If rectification leaves the principal points at different columns, a point at infinity no longer has zero disparity. The Middlebury 2014 data supplies the offset as doffs $= c_{x1} - c_{x0}$ (right minus left principal-point column) and converts with $Z = fB/(d + \text{doffs})$, so raw disparities of distant points can be zero or negative.

Properties

• Search range. Matching tests candidates between $d_\text{min}$ and $d_\text{max} = fB/Z_\text{min}$, set by the nearest depth of interest, so the range, and the size of the cost volume, grows with baseline, focal length, and resolution.
• Uniform in disparity, non-uniform in depth. A fixed disparity error $\Delta d$ gives a depth error $\Delta Z \approx Z^2 \Delta d/(fB)$, growing with the square of depth, as derived in stereo vision.
• Sub-pixel values. Integer disparities make the scene look like a stack of thin layers, so many algorithms refine them to fractional values, for example by fitting a curve to the matching costs around the best integer disparity (Scharstein and Szeliski, 2002). Learned matchers regress fractional disparities directly, for example with the soft argmin (see cost volume).
• Invalid pixels. Half-occluded pixels, seen in only one image, pixels whose match leaves the other image, and pixels a matcher rejects (for instance by a left-right check) have no valid disparity. Ground truth has gaps too: KITTI’s comes from accumulated, uninterpolated laser scans and covers about 50% of pixels on average (Geiger, Lenz, and Urtasun, 2012).

Inverse depth as a parametrization

Inverse depth $\rho = 1/Z$ is often used as the unknown even without a stereo rig, because it stays finite for distant points ($\rho = 0$ at infinity). Okutomi and Kanade (1993) expressed the matching costs of several stereo pairs as functions of inverse distance, shared across baselines, so the costs could be summed. In monocular SLAM, Civera, Davison, and Montiel (2008) parametrized each new point by its inverse depth relative to the camera that first saw it, which keeps the measurement equations close to linear and lets distant, low-parallax points, whose depth uncertainty is far from Gaussian, be used from their first observation. In monocular depth estimation, Ranftl et al. (2022) trained in disparity space, inverse depth up to unknown scale and shift, partly because some training datasets provide only such relative disparity.

Examples

Storing disparity maps

Each benchmark fixes its own encoding, and misreading one gives silently wrong depths:

• KITTI. The 2012 and 2015 development kits store disparity as a 16-bit unsigned PNG. Dividing the stored value by 256 gives disparity in pixels, and 0 marks a pixel without a value. The evaluation code fills gaps in submitted maps before scoring, row by row, with the smaller (background) of the two neighboring valid disparities.
• Middlebury. Version 3 of the benchmark stores floating-point disparities in PFM files, with infinity marking unknown values, and each 2014 scene’s calibration file supplies $f$, $B$ (in millimeters), doffs, and ndisp, a conservative bound that lets algorithms search $d = 0, \dots, \text{ndisp} - 1$.
• OpenCV. The StereoBM and StereoSGBM matchers return 16-bit fixed-point disparities with 4 fractional bits, so values must be divided by 16; invalid pixels are set to $16(d_\text{min} - 1)$, one step below the minimum disparity.

The code round-trips a small map through the KITTI encoding, converts it to depth for an illustrative rig, and shows the encoding’s limits.

import numpy as np
np.set_printoptions(suppress=True)   # fixed-point rather than scientific notation

f, B = 720.0, 0.54   # focal length (px) and baseline (m) of an illustrative rectified rig

# A 2 x 4 disparity map in pixels; inf marks pixels with no disparity (Middlebury-style).
disp = np.array([[64.25, 32.0, 8.5, np.inf],
                 [ 4.0,   1.0, 0.3, 0.0]])

# KITTI-style encoding: uint16 = round(256 * d); 0 means "invalid", so a valid 0 is stored as 1.
valid = np.isfinite(disp) & (disp >= 0)
png = np.zeros(disp.shape, np.uint16)
png[valid] = np.maximum(np.round(disp[valid] * 256), 1).astype(np.uint16)
print("stored uint16:\n", png)

# Decoding, as in the KITTI development kit: d = I / 256, valid where I > 0.
d = png.astype(np.float64) / 256.0
ok = png > 0
Z = np.full(d.shape, np.nan)
Z[ok] = f * B / d[ok]
print("depth (m):\n", np.round(Z, 2))

# The encoding quantizes disparity to 1/256 px and caps it just below 256 px.
print(f"largest storable disparity: {65535 / 256:.3f} px "
      f"-> nearest depth {f * B / (65535 / 256):.3f} m")
for d0 in [64.0, 8.0, 1.0]:
    Z0 = f * B / d0
    step = Z0**2 / (f * B) / 256          # depth change for one 1/256 px step
    print(f"d = {d0:5.1f} px: Z = {Z0:6.2f} m, one quantization step = {100 * step:.3f} cm")

Output:

stored uint16:
 [[16448  8192  2176     0]
 [ 1024   256    77     1]]
depth (m):
 [[    6.05    12.15    45.74      nan]
 [   97.2    388.8   1292.63 99532.8 ]]
largest storable disparity: 255.996 px -> nearest depth 1.519 m
d =  64.0 px: Z =   6.08 m, one quantization step = 0.037 cm
d =   8.0 px: Z =  48.60 m, one quantization step = 2.373 cm
d =   1.0 px: Z = 388.80 m, one quantization step = 151.875 cm

Zero cannot be stored as a valid disparity, since it means “no value”; the development kit’s writers store a valid zero as 1, that is 1/256 pixel, which decodes to a depth of about 100 km here. The 1/256-pixel step is negligible at 64 pixels, but at 1 pixel it moves the depth by 1.5 m, and the 0.3-pixel value decodes to 1293 m instead of 1296 m. The 16-bit range also caps disparity just below 256 pixels, so on this rig nothing closer than about 1.5 m can be stored.

Common Misconceptions

• “Disparity is depth.” It is in pixels and inversely proportional to depth; converting it needs $f$, $B$, and any principal-point offset.
• “Averaging disparities averages depths.” Since $Z = fB/d$ is nonlinear, filtering or interpolating in disparity and then converting differs from doing so in depth, most at small disparities.
• “A stored zero is a measurement.” In KITTI files it means “no value”; read as a disparity it would put the point at infinity.

Where It Is Used

• Stereo matching produces disparity maps over a cost volume indexed by disparity.
• Depth maps and point clouds follow from $Z = fB/d$, a special case of triangulation.
• Benchmarks such as KITTI and Middlebury define ground truth and error metrics in disparity.
• Stereo rectification makes disparity a purely horizontal, one-dimensional quantity.
• Monocular depth estimation and SLAM use inverse depth or relative disparity as the estimated variable.