High Dynamic Range Imaging

High dynamic range (HDR) imaging records scenes whose brightest and darkest parts differ by more than a single exposure can capture, and then shows them on media whose range is smaller still. A room with a sunlit window, a tunnel exit or a lamp in a dark street holds radiances many thousands of times apart. One exposure either clips the bright parts or buries the dark parts in noise. HDR imaging measures the scene in pieces, usually as several exposures, and combines them into a radiance map: an image of floating-point values proportional to scene radiance (Debevec and Malik, 1997). Tone mapping then compresses that map into an image a display can show.

How a single frame is exposed, and why every setting trades one end of the range against the other, is the subject of exposure; this article covers what lies beyond one frame. HDR merging is also one technique of computational photography, the wider field of burst, multi-frame and coded capture.

Purpose

Three different dynamic ranges, ratios between the largest and the smallest value that matters, meet in HDR imaging.

• Scene range. The ratio between the brightest and the darkest radiance in the part of the scene that matters. Onzon, Mannan and Heide (2021) put the range a vision system can meet, from no ambient light at night to bright sunlight, at about 280 dB (“up to” in the abstract, “can exceed” in Section 1), and the range found at once in a typical outdoor scene at 120 dB, with cases such as a tunnel exit holding the sun and near-darkness in one view (Section 1).
• Sensor range. The ratio between the signal at which the sensor saturates and the smallest signal it can distinguish from noise, defined and measured as described under image sensor; Onzon et al. (2021, Section 1) give 60–70 dB for conventional CMOS sensors in a single capture. Optical noise such as veiling glare, stray light and aperture ghosts lowers the range a camera reaches in practice, and Onzon et al. (2021, Section 3) note that it dominates sensor noise in HDR cameras.
• Output range. Images stored for display usually take only 256 values per channel, as Mann and Picard (1995) point out; Debevec and Malik (1997, Section 1) note that cameras often convert a 12-bit converter output to 8 bits through a nonlinear curve that clips everything above a saturation point.

HDR imaging closes two gaps: between the scene and the sensor, by capturing more range than one frame holds, and between the captured range and the output, by tone mapping. The first gap matters to any system that measures light; the second matters only when a person looks at the result.

When to Use It

Bracketing and merging suit scenes whose important parts span more stops than the sensor’s dynamic range and which stay still for the burst: architecture, interiors with windows, product and inspection set-ups with specular parts, and image-based lighting, where the light sources themselves must be measured. Debevec and Malik (1997, Section 1.1) argue that image processing in general benefits, because blurring, edge detection, colour correction and correspondence expect values proportional to radiance, and clipping breaks that.

It is the wrong tool when the scene fits the sensor, when exposure control alone can place the important part of the range, or when things move fast compared with the burst, where sequential frames produce ghosts. Single-shot HDR sensors avoid the burst at other costs, described under Variants. When only a pleasing picture is needed, exposure fusion skips the radiance map altogether.

How It Works

Exposure bracketing

The camera records the same scene several times with different exposure, usually by changing the shutter time. Debevec and Malik (1997, Section 2.1) rely on reciprocity: the response depends only on the exposure $X = E\,\Delta t$, the product of the sensor irradiance $E$ and the exposure time $\Delta t$, which holds for a sensor that counts all the photons it absorbs during the exposure. They advise varying the time rather than the aperture, because the aperture also changes vignetting (Debevec and Malik, 1997, Section 2.1, footnote 4); Kirk and Andersen (2006, Section 2.1) add the change in depth of field.

Neighbouring exposures must overlap: some pixels must fall in the useful middle of the response in both, or nothing ties the two exposures together (Debevec and Malik, 1997, Section 2.3). The shortest exposure has to keep the brightest region that matters below saturation, and the longest has to lift the darkest region well above the noise; the bracket’s span in stops then has to cover the scene’s range (this rule follows from the definitions above).

The camera response function

A RAW sensor value minus its black level is close to proportional to exposure, but the values a camera stores usually are not. The camera response function $f$ maps exposure to pixel value, $Z = f(X)$; in film it is the characteristic curve, and in a digital camera it combines gamma, a tone curve, quantization to 8 bits and clipping at saturation, the last steps of image formation. Debevec and Malik (1997, Section 3) found that a Kodak DCS460 remapped its linear sensor data nonlinearly, presumably to mimic film.

Writing $Z_{ij}$ for pixel $i$ in exposure $j$ and assuming $f$ is monotonic, Debevec and Malik (1997, Section 2.1) invert and take logarithms:

$$g(Z_{ij}) = \ln E_i + \ln \Delta t_j, \qquad g = \ln f^{-1}.$$

The pixel values and the exposure times are known; the irradiances $E_i$ and the function $g$ are not. Because $Z$ takes only 256 values, $g$ is a table of 256 unknowns.

Recovering the response: Debevec and Malik’s method

Debevec and Malik (1997, Section 2.1) find $g$ and the $\ln E_i$ that best satisfy all these equations in the least-squares sense, with a penalty on the second difference of $g$:

$$\mathcal{O} = \sum_{i=1}^{N} \sum_{j=1}^{P} \left\{ w(Z_{ij}) \left[ g(Z_{ij}) - \ln E_i - \ln \Delta t_j \right] \right\}^2 + \lambda \sum_{z=Z_{\min}+1}^{Z_{\max}-1} \left[ w(z)\, g''(z) \right]^2,$$

where $g''(z) = g(z-1) - 2g(z) + g(z+1)$ and $w$ is a hat function that rises linearly from $Z_{\min}$ to the middle code and falls back to zero at $Z_{\max}$:

$$w(z) = \begin{cases} z - Z_{\min}, & z \le \tfrac{1}{2}(Z_{\min} + Z_{\max}), \\ Z_{\max} - z, & z > \tfrac{1}{2}(Z_{\min} + Z_{\max}). \end{cases}$$

The hat de-emphasizes the ends of the curve, where $g$ is steep and fits poorly, and it removes clipped values from the fit. The objective is quadratic in all unknowns, so minimizing it is a linear least-squares problem; the paper says the authors solve it with the singular value decomposition, while the gsolve.m routine printed in the paper’s appendix, under a comment that says SVD, calls MATLAB’s backslash solver. Three details complete the method (Debevec and Malik, 1997, Section 2.1):

• Scale. Adding a constant to $g$ and to every $\ln E_i$ leaves the objective unchanged, so the curve is fixed by $g(Z_{\text{mid}}) = 0$. The radiance map is therefore known only up to a scale factor, one per colour channel when channels are solved separately, which changes the colour balance (Section 2.6).
• Sampling. Using every pixel makes the system needlessly large. With $N$ sampled pixels and $P$ photographs, the system is sufficiently overdetermined when $N(P-1) > Z_{\max} - Z_{\min}$; for a range of 255 and 11 photographs, about 50 pixels are “more than adequate”. The pixels should spread over the range of values and over the image and come from smooth regions; the authors chose them by hand.
• Monotonicity is not enforced; the smoothness term kept the curve monotonic in the authors’ experience.

$\lambda$ trades fidelity to noisy data against smoothness and should be chosen for the noise expected in the pixel values. The method needs known exposure times; Debevec and Malik checked their film camera’s shutter against audio recordings (Section 2.1, footnote 4).

An earlier and a later formulation relax different assumptions. Mann and Picard (1995, Section 4) modelled the response parametrically as $f(q) = \alpha + \beta q^{\gamma}$ and estimated it by comparing pairs of differently exposed images, through the joint histogram of their values; Debevec and Malik (1997, Section 1.2) criticized the fixed parametric form and the treatment of noise. Mitsunaga and Nayar (1999, Sections 3–4) fit a polynomial $f^{-1}$ instead, which lets them estimate the exposure ratios as well, starting from rough guesses such as the f-numbers on an inexpensive lens. Their polynomial model limits an ambiguity in which the response and the exposure ratios can only be recovered up to an unknown exponent. On 100 randomly generated response curves with four noisy synthetic images each, they report a worst-case error under 2.7% (Section 5).

Merging into a radiance map

With $g$ known, each exposure gives an estimate $\ln E_i = g(Z_{ij}) - \ln \Delta t_j$ of the pixel’s log irradiance, and Debevec and Malik (1997, Section 2.2) average them with the same hat weights:

$$\ln E_i = \frac{\sum_{j=1}^{P} w(Z_{ij}) \left( g(Z_{ij}) - \ln \Delta t_j \right)}{\sum_{j=1}^{P} w(Z_{ij})}.$$

Averaging reduces noise, and because the weight ignores saturated values, blooming has little effect on the map. The choice of weights is where methods differ. Kirk and Andersen (2006, Table 1, Section 2.2) review the main schemes, each with its rationale:

• Hat weights (Debevec and Malik), in the log domain, shaped after the expected derivative of the response.
• Certainty weights $f'$, the slope of the response at the observed value (Mann and Picard, 1995), which suppress the flat ends of a film-like curve but weight all values equally for a linear camera.
• Signal-to-noise weights, the linearized value divided by its slope, $f^{-1}(Z)/(f^{-1})'(Z)$ (Mitsunaga and Nayar, 1999, Eq. 14, where $f$ denotes the inverse response), derived from the signal-to-noise ratio under the assumption that the measurement noise does not depend on the pixel value.
• Inverse-variance weights. Under a noise model whose variance grows linearly with the signal plus a constant term, the minimum-variance estimate weights each exposure’s radiance estimate by the inverse of its variance; for a linear camera this is $t_j^2 / (A d_{ij} + \sigma_c^2)$, with $d_{ij}$ the pixel value, $A$ the sensor gain and $\sigma_c^2$ the read and quantization noise (Kirk and Andersen, 2006, Section 2.3).

Inverse-variance weighting is the noise-optimal choice when the noise model is right; for a linear camera, Kirk and Andersen find the signal-to-noise weights almost equivalent (Sections 2.3 and 4). For a shot-noise-limited pixel the variance of the count grows with the count, so the variance of $d_{ij}/t_j$ falls as $1/t_j$ and the weight grows with exposure time: the longest unsaturated exposure deserves most of the weight (this follows from the Poisson model in image noise). A hat function weights by code value alone, so a pixel keeps a sizeable weight in a short, noisy exposure as long as its code is not near zero; the example below shows the cost.

Alignment and ghost removal

Merging assumes that each pixel sees the same radiance in every exposure, which camera shake and moving objects break. Misalignment and motion leave ghosts: semi-transparent copies of moving objects, or tearing where an alignment failed. Kalantari and Ramamoorthi (2017, Section 2) sort the remedies into three groups:

• Rejecting moving pixels. After a global registration, pixels whose values are inconsistent across the stack are detected (by predicting each exposure from another, by local entropy, by gradients or by bitmaps) and given low weight. Such methods cannot handle moving content that is itself of high dynamic range, because they discard it.
• Aligning before merging. Frames are aligned to a reference by a translation (Ward, 2003, in their review), a homography, or dense optical flow for non-rigid motion, then merged. Registration errors appear as ghosting and tearing in the result (Section 1).
• Joint alignment and reconstruction. Sen et al. (2012) formulated HDR reconstruction as patch-based energy minimization, through what they call the HDR image synthesis equation, so that the result is aligned with one reference exposure yet draws information from all of them.

Kalantari and Ramamoorthi (2017, Sections 3–4) kept flow alignment but replaced the merge by a convolutional neural network that learns to hide alignment artifacts. They built training data by photographing each scene as two brackets of three exposures, one static and one with a moving subject: the static bracket gives the ground-truth radiance map, and its middle exposure replaces the dynamic bracket’s, so the reference matches the ground truth (Section 4); there are 74 training scenes. On their 15 test scenes, their three architectures reached 42.74–43.26 dB PSNR on tonemapped images, against 40.75 dB for Sen et al.’s method and 39.10 dB for Kang et al.’s flow-based method (Table 1).

Tone Mapping

A radiance map holds more range than a display, so it has to be compressed for viewing. Debevec and Malik (1997, Section 3) displayed their first radiance map by taking the logarithm of the values and mapping that range linearly to the display. Tone-mapping operators are usually classified as global or local (Durand and Dorsey, 2002, Section 2).

• Global operators apply the same curve to every pixel. Their main advantage is speed (Mertens, Kautz and Van Reeth, 2007, Section 2), and Durand and Dorsey note that most do not directly address contrast reduction; the example below shows that one curve cannot keep contrast in both the shadows and the highlights.
• Local operators let the mapping vary across the image, reducing large-scale contrast while keeping local detail. Reinhard et al. (2002) designed an operator after the techniques Ansel Adams developed for photographic printing; Mertens et al. (2007, Section 2) describe it as rescaling pixel values by a multi-scale measure related to contrast. Durand and Dorsey (2002) split the log luminance into a base layer, computed with the bilateral filter, and a detail layer, and compress only the base. The usual failure of local operators is a halo, a bright or dark fringe along strong edges, which an edge-preserving base layer reduces (Durand and Dorsey, 2002, Section 2).

Mitsunaga and Nayar (1999, Section 7) displayed their radiance maps with histogram equalization, applied to the whole image and, for close-ups, within small windows, a mapping built from the image histogram.

Exposure fusion (Mertens et al., 2007) skips the radiance map. It blends the bracketed images directly, with a weight per pixel and exposure that multiplies three quality measures: contrast, the absolute response of a Laplacian filter; saturation, the standard deviation of the three colour channels; and well-exposedness, a Gaussian of each channel’s value around 0.5 with $\sigma = 0.2$ (Section 3.1). Blending the images with these weights directly leaves seams where weights change abruptly, so the authors blend each level of a Laplacian image pyramid of the inputs with a Gaussian pyramid of the weights (Section 3.2). The method needs neither the response curve nor the exposure times and accepts a flash image in the stack, but, as the authors note, it is not physically based and does not extend the dynamic range of the pictures; it produces a display image directly (Section 1). It suits a picture, not a measurement.

How to Apply It

A practical sequence for a static scene, following the sections above:

1. Fix focus, aperture, white balance and gain, and vary only the shutter time; use RAW or a linear mode when the camera has one.
2. Choose the bracket as described above. Debevec and Malik (1997, Section 2.3) suggest recovering a response curve once, from three or four exposures one or two stops apart, and reusing it.
3. For processed images, recover the response and check that the curve is smooth and monotonic.
4. Align the frames and deal with moving objects, then merge in floating point with weights that zero out clipped and black values and, given a noise model, inverse-variance weights.
5. Keep the radiance map in floating point; tone-map only the copy that people look at.

Example: response recovery, merging and tone curves

The example renders a room whose brightness rises from left to right and a window much brighter still, both covered by vertical stripes of ±20% that serve as detail. A simulated camera records it at five shutter times two stops apart, with photon shot noise, 5 electrons of read noise, a 20,000-electron full well, a gamma curve followed by a gentle S-shaped tone curve, and 8-bit output. The code recovers $g$ with its own implementation of Debevec and Malik’s least-squares system on 300 random pixels, compares the result with the true inverse response, merges the exposures with hat weights and with inverse-variance weights, and compares single exposures, the radiance map and three global tone curves.

import numpy as np

# --- 1. A rendered high-contrast scene: dark room, bright window -----------
H, W = 120, 180
yy, xx = np.mgrid[0:H, 0:W]
stripes = 1 + 0.2 * np.sign(np.sin(2 * np.pi * xx / 12))  # +/-20% detail
L = (0.2 + 1.8 * xx / W) * stripes                         # room: about 0.2-2.4
window = (yy > 20) & (yy < 70) & (xx > 100) & (xx < 160)
L[window] = (300 + 2400 * (70 - yy[window]) / 50) * stripes[window]
room = ~window
print(f"scene radiance {L.min():.2f} to {L.max():.0f}: "
      f"{L.max() / L.min():.0f}:1, {np.log2(L.max() / L.min()):.1f} stops")

# --- 2. Camera: shot + read noise, full well, nonlinear response, 8 bits ---
full_well, k_e, read = 20000.0, 5000.0, 5.0      # electrons; e- per unit L*t
def crf(x):                       # x in [0,1] -> [0,1]: gamma, then an S-curve
    u = x ** (1 / 2.2)
    return u - 0.06 * np.sin(2 * np.pi * u)

def capture(L, t, rng):
    e = rng.poisson(k_e * L * t) + rng.normal(0, read, L.shape)
    x = np.clip(e / full_well, 0, 1)              # clipping at the full well
    return np.round(255 * crf(x)).astype(int)     # 8-bit output

times = 4.0 ** -np.arange(1, 6)                  # 1/4 ... 1/1024 s, 2 stops apart
B = np.log(times)

# --- 3. Debevec & Malik: solve for g = ln f^-1 by linear least squares ------
w = np.minimum(np.arange(256), 255 - np.arange(256)).astype(float)  # hat weights

def gsolve(Z, B, lam, w):
    n, (N, P) = 256, Z.shape
    A = np.zeros((N * P + n - 1, n + N)); b = np.zeros(A.shape[0])
    k = 0
    for i in range(N):                            # data: w[g(Z) - lnE - ln t]
        for j in range(P):
            wij = w[Z[i, j]]
            A[k, Z[i, j]], A[k, n + i], b[k] = wij, -wij, wij * B[j]; k += 1
    A[k, 128] = 1; k += 1                         # fix the scale: g(128) = 0
    for z in range(1, n - 1):                     # smoothness: sqrt(lam) * w * g''
        A[k, z - 1:z + 2] = np.sqrt(lam) * w[z] * np.array([1, -2, 1]); k += 1
    x = np.linalg.lstsq(A, b, rcond=None)[0]
    return x[:n], x[n:]

# true g, for checking: invert the response at each code value
xs = np.linspace(1e-6, 1, 200001)
g_raw = np.log(np.interp(np.arange(256) / 255, crf(xs), xs) * full_well / k_e)
g_true = g_raw - g_raw[128]
X128 = np.exp(g_raw[128])  # scale to electrons, which calibration would supply

def merge(imgs, g):
    # (a) Debevec & Malik: hat-weighted mean of g(Z) - ln t, in the log domain
    wz = w[imgs]
    hat = np.exp((wz * (g[imgs] - B[:, None, None])).sum(0) / wz.sum(0))
    # (b) inverse-variance weights in the linear domain, from a noise model
    e = k_e * X128 * np.exp(g[imgs])              # electrons, per exposure
    var_e = e + read**2 + (e * np.gradient(g)[imgs])**2 / 12   # shot+read+quant
    t = times[:, None, None]
    wv = np.where((imgs > 0) & (imgs < 255), (k_e * t)**2 / var_e, 0)
    ivw = (wv * e / (k_e * t)).sum(0) / wv.sum(0)
    return hat, ivw

def rel_err(E):                    # |relative error|, after fitting the scale
    r = np.log(E) - np.log(L)
    return np.abs(np.expm1(r - np.median(r)))

def run(seed, lam):
    rng = np.random.default_rng(seed)
    imgs = np.stack([capture(L, t, rng) for t in times])     # (P, H, W)
    flat = imgs.reshape(len(times), -1).T                    # (H*W, P)
    idx = rng.choice(H * W, 300, replace=False)              # 300 sample pixels
    g, _ = gsolve(flat[idx], B, lam, w)
    return g, imgs

print("\nerror of g (natural-log units), seed 0: rms over codes 10-250, max over 1-9")
for lam in [0.01, 1, 100, 10**4, 10**6]:
    g, _ = run(0, lam)
    print(f"  lambda={lam:<8} rms {np.sqrt(np.mean((g - g_true)[10:251]**2)):.3f}"
          f"   max below code 10: {np.abs(g - g_true)[1:10].max():.3f}")

print("\nlambda=100, seeds 0-9: median |relative radiance error| room, window; 99th pct")
print("            hat weights (log domain)     inverse variance (linear)")
stats = []
for seed in range(10):
    g, imgs = run(seed, 100)
    row = []
    for E in merge(imgs, g):
        r = rel_err(E)
        row += [np.median(r[room]), np.median(r[window]), np.percentile(r, 99)]
    stats.append(row)
    print(f"  seed {seed}:  " + "  ".join(f"{100*v:5.2f}%" for v in row))
st = 100 * np.array(stats)
print("  range:   " + "  ".join(f"{a:.1f}-{b:.1f}" for a, b in zip(st.min(0), st.max(0))))

# --- 4. Single exposures against the radiance map, then global tone curves -
g, imgs = run(0, 100)
E = merge(imgs, g)[0]
print("\nclipped (255) or black (0) pixels per exposure, room / window:")
for t, Z in zip(times, imgs):
    print(f"  t=1/{1/t:<5.0f} room {100*np.mean((Z[room]==0)|(Z[room]==255)):5.1f}%"
          f"  window {100*np.mean((Z[window]==0)|(Z[window]==255)):5.1f}%")

def stripe_codes(img, region):     # mean code on bright minus dark stripes
    on = region & (stripes > 1)
    off = region & (stripes < 1)
    return img[on].mean() - img[off].mean()

lin = np.round(255 * E / E.max())                                # linear scaling
lE = np.log(E)
logc = np.round(255 * (lE - lE.min()) / (lE.max() - lE.min()))  # log curve
Ls = 0.18 * E / np.exp(lE.mean())                # scale log-average to 0.18
comp = np.round(255 * (Ls / (1 + Ls)) ** (1 / 2.2))              # L/(1+L)
print("\nstripe detail in 8-bit codes (bright minus dark stripes), room / window:")
for name, img in [("single exposure t=1/64", imgs[2]),
                  ("single exposure t=1/1024", imgs[4]),
                  ("radiance map, linear scaling", lin),
                  ("radiance map, log curve", logc),
                  ("radiance map, L/(1+L) curve", comp)]:
    print(f"  {name:30s} {stripe_codes(img, room):6.1f}  {stripe_codes(img, window):6.1f}")

Output (NumPy 2.3.5; identical on two runs; about 1.5 s):

scene radiance 0.20 to 3182: 15912:1, 14.0 stops

error of g (natural-log units), seed 0: rms over codes 10-250, max over 1-9
  lambda=0.01     rms 0.095   max below code 10: 2.544
  lambda=1        rms 0.122   max below code 10: 2.464
  lambda=100      rms 0.035   max below code 10: 2.097
  lambda=10000    rms 0.029   max below code 10: 2.530
  lambda=1000000  rms 0.520   max below code 10: 4.662

lambda=100, seeds 0-9: median |relative radiance error| room, window; 99th pct
            hat weights (log domain)     inverse variance (linear)
  seed 0:   2.99%   2.96%  19.00%   2.03%   3.20%   9.93%
  seed 1:   3.09%   3.64%  21.58%   2.15%   3.52%  10.14%
  seed 2:   3.03%   1.90%  19.91%   2.06%   1.91%   9.83%
  seed 3:   2.97%   2.31%  18.54%   1.93%   2.35%   9.79%
  seed 4:   3.12%   2.40%  21.12%   2.19%   2.45%  10.60%
  seed 5:   3.16%   2.46%  21.29%   2.21%   2.58%  10.72%
  seed 6:   2.91%   1.12%  19.98%   1.87%   0.93%   9.77%
  seed 7:   3.90%   2.65%  21.74%   3.45%   2.09%  12.11%
  seed 8:   3.13%   2.50%  19.84%   2.40%   2.39%  10.45%
  seed 9:   3.04%   3.29%  19.42%   2.00%   3.60%   9.78%
  range:   2.9-3.9  1.1-3.6  18.5-21.7  1.9-3.5  0.9-3.6  9.8-12.1

clipped (255) or black (0) pixels per exposure, room / window:
  t=1/4     room   0.0%  window 100.0%
  t=1/16    room   0.0%  window 100.0%
  t=1/64    room   0.0%  window 100.0%
  t=1/256   room   3.2%  window  68.0%
  t=1/1024  room  21.1%  window   0.0%

stripe detail in 8-bit codes (bright minus dark stripes), room / window:
  single exposure t=1/64            2.1     0.0
  single exposure t=1/1024          0.5    32.9
  radiance map, linear scaling      0.0    44.7
  radiance map, log curve           7.9     9.8
  radiance map, L/(1+L) curve      10.1     0.4

The code multiplies the smoothness rows by $\sqrt{\lambda}$, so that $\lambda$ means what it does in the objective; the paper’s MATLAB routine multiplies them by its parameter directly, which then plays the role of $\sqrt{\lambda}$. The smoothness weight behaves as the objective predicts. With $\lambda$ of 0.01 or 1 the curve follows the noise in the sampled pixels; at 100 and $10^4$ its root-mean-square error over codes 10 to 250 is about 0.03 in log units, about 3% in exposure; at $10^6$ the penalty flattens the curve’s bends and the error grows more than tenfold. Below code 10 no setting recovers the curve: the darkest codes are dominated by read noise and quantization, and the hat gives them almost no weight in the fit. That costs little in the merge, which gives them almost no weight either.

The merge recovers radiance over 14 stops, which no single exposure holds: the three longest exposures clip the whole window, the shortest leaves a fifth of the room black, and the 1/256 s frame still clips two thirds of the window. The two weightings then differ where the noise model predicts. The dark end of the room collects only a few hundred electrons even in the longest exposure, and hat weights also lend substantial weight to the same pixels in shorter, noisier exposures, whose codes sit on the rising side of the hat. Inverse-variance weights favour the long exposure, which lowers the median error in the room and halves the 99th percentile over the ten seeds. The window’s error, by contrast, changes from seed to seed by similar amounts under both weightings. Both merges use the same recovered curve, so this suggests the window’s error comes from the curve, an error in $g$ at high codes that shifts the window relative to the room, rather than from noise.

The last table shows why a radiance map is not yet a picture. Each single exposure shows the stripes in one region only. Scaling the radiance map linearly to 8 bits keeps the window’s stripes and maps the whole room to the lowest codes. A logarithmic curve gives the ±20% stripes about the same number of codes in both regions, because equal ratios become equal differences, but it spends codes evenly on all 14 stops, so no region gets much contrast. The compressive curve $L/(1+L)$, applied after scaling the log-average to 0.18, gives the room more contrast and leaves the window’s stripes less than one code apart. No single global curve serves both regions here, which is the case for local operators.

Variants

HDR sensors and single-shot capture

Sensors can extend the range inside one frame, by several exposures or gains per frame, or by a nonlinear pixel response. Onzon et al. (2021, Section 1) name sequential multi-capture and pixels split into two sub-pixels as the most successfully deployed architectures, report up to 140 dB for modern HDR sensors, and list their costs: motion artifacts when exposures differ in length or start, lower fill factor and resolution, and, for interleaved designs, no global shutter.

• Several exposures in one frame. Exposure times can vary over the sensor. As of October 2026, Basler’s documentation describes cameras that expose alternate rows with two times that start together, cameras that set four exposure times within each 2 × 2 block of pixels, and cameras that set long and short exposures in a checkerboard and combine them inside the camera; others cycle exposure settings frame by frame with a sequencer for merging in software (Basler, HDR). Eilertsen et al. (2017, Section 2.1) list the costs of such single-shot schemes as custom systems, calibration problems or lower resolution.
• Several gains. A pixel’s conversion gain sets the trade between read noise and the charge that fits before the output clips. Basler’s documentation (as of October 2026) describes a switchable mode in which high conversion gain lowers read noise for dark scenes at the cost of dynamic range, and low conversion gain raises the saturation capacity (Basler, Conversion Gain Mode). Reading pixels at two analog gains, for example alternating every two rows, gives a single-shot HDR image without the motion artifacts of sequential exposures (Hajisharif, Kronander and Unger, 2015, Section 1).
• Split pixels and overflow capacitors. A split pixel holds a large and a small photodiode, the large one for low light and the small one for bright light, and combines their signals; a pixel with a lateral overflow integration capacitor (LOFIC) keeps one photodiode and stores the charge that overflows it once it is full. Both give an HDR image within one exposure. Li et al. (2024), engineers at onsemi who modelled both pixel types with a monochrome CMOS sensor, found that the split pixel’s sharpness changes with light level, because the two photodiodes sample different areas, and that the sparse small photodiodes produce jagged edges, while the single-photodiode pixel showed neither.
• Multi-slope (knee-point) response. Some sensors reset pixels to a set level, or change their capacity, at set times during the exposure. The result is one image whose response is made of linear segments meeting at knee points; the GenICam naming convention defines camera features to configure them (SFNC 2.8, Section 5.8).
• Logarithmic pixels. A pixel whose output follows the logarithm of the light compresses the range at capture. Event cameras use photoreceptors that operate on a logarithmic scale, and Gallego et al. (2022, Section 2.2 of the arXiv version) attribute their range of more than 120 dB, against about 60 dB for high-quality frame cameras, to that and to each pixel working independently.

Nonlinear responses also change how a sensor is characterized: EMVA 1288 Release 4 Linear applies only to cameras with a linear response, and the separate Release 4 General covers the rest (EMVA 1288, Section 1).

Single-image HDR reconstruction

When only one ordinary image exists, the clipped parts have to be inferred, a problem also called inverse tone mapping. Eilertsen et al. (2017, Sections 1 and 3) designed a CNN they call a hybrid dynamic range autoencoder: an encoder for the LDR input, a decoder that predicts HDR values in the log domain, and skip connections between them. They trained it on HDR images from which they simulated saturated LDR inputs for a range of cameras, after pre-training on HDR data simulated from a subset of the MIT Places database. The network’s output replaces the linearized input only near saturation, through a blend that ramps in from 95% of the maximum value, so unclipped pixels are kept; the authors state that this works when the saturated areas are reasonably small (Section 1), and they evaluated the results in a subjective experiment on an HDR display. They describe the concurrent work of Endo et al. as predicting a set of shorter and longer exposures from the single image and then merging them (Section 2). Liu et al. (2020) model the camera pipeline explicitly, as clipping, a nonlinear response and quantization, and train three networks to undo these steps, add a refinement network and fine-tune them jointly; they report HDR-VDP-2 scores on four datasets. In both cases the content of clipped regions is a plausible prediction, not a measurement (our reading), which matters when the result feeds a measurement.

HDR in Vision Systems

For a vision system, the goal of HDR is usually not a pleasing picture. Mitsunaga and Nayar (1999, Section 1) observe that most vision algorithms assume, explicitly or not, that measured brightness is linear in scene radiance. A linear radiance map preserves that assumption over the whole range, while a tone-mapped image does not: tone mapping is a nonlinear curve, local operators vary it across the image, and a scene-dependent operator changes it from frame to frame, with the consequences for brightness constancy that digital camera describes (our reasoning from the definitions).

Onzon et al. (2021) take the alternative route of controlling a low-dynamic-range sensor better. They train an exposure-control network end to end with a differentiable image-processing pipeline and an object detection network, simulating low-dynamic-range captures from HDR data recorded with an automotive HDR sensor, and report that their method outperforms conventional auto-exposure by 6.6 points of mean average precision across their automotive scenarios (Section 1; the abstract says more than 6 points). In our assessment neither route dominates: HDR merging adds artifacts on moving objects, and exposure control loses whatever the chosen exposure clips.

Best Practices

• Prefer linear data. A linear camera removes the uncertainty of the response curve; most machine-vision cameras can operate linearly, and many consumer cameras can store linear RAW data (Kirk and Andersen, 2006, Section 1). It also lets inverse-variance weights use a calibrated noise model, which the photon transfer measurement described under image sensor supplies.
• Check the result against the inputs. Re-exposing the radiance map through the recovered curve, as Debevec and Malik (1997, Section 2.7) do with their virtual photographs, should reproduce each input exposure wherever it was not clipped.

Pitfalls

• Merging processed images without linearizing them, or with a curve that scene-dependent processing changed between frames, which no single recovered curve can fit.
• Highlights clipped in the shortest exposure, or shadows black in the longest, cannot be recovered by merging; only the bracket or the sensor can fix them.
• Too much or too little smoothing of the response: the example shows the error growing at both ends of $\lambda$.
• Glare. The optical noise noted under Purpose spreads light from bright regions into dark ones, so the darkest values in a radiance map can be glare rather than scene radiance (our reasoning).
• Evaluating in one domain only. Kalantari and Ramamoorthi (2017, Section 5) report PSNR on tonemapped images, which they argue better reflects what a viewer sees, alongside PSNR on linear values; in their Table 1 the two orders differ, with Kang et al.’s method ahead of Sen et al.’s on linear values and behind on tonemapped ones.

Evidence and Impact

Debevec and Malik (1997, Section 1.2) describe Madden (1993) as combining several CCD integration times when the sensor’s linear output was available, and Mann and Picard (1995) as addressing an unknown, nonlinear response with what they considered a limited method. Their own least-squares method, with known exposure times and no parametric model, recovered a radiance map with “over four orders of magnitude” of useful range from a digital camera (Section 3, Fig. 5). Twenty years on, Kalantari and Ramamoorthi (2017, Section 1) and Liu et al. (2020, Section 1) still call merging bracketed exposures the most common way to obtain an HDR image.