Digital Camera

A digital camera is a complete imaging system: a lens that forms an image, an aperture and a shutter that decide how much light reaches it and when, an image sensor that turns that light into numbers, and an image signal processor (ISP) that turns those numbers into a picture. Each part decides something a vision method assumes about its input: where pixels are, when they were exposed, whether their values are proportional to light, and what their noise looks like. Most surprises come from the last stage, because a camera built for photography processes its data to look good rather than to measure.

The geometry of projection is the subject of the pinhole camera model, camera parameters and camera calibration; the photometry of the chain is followed in image formation; and the choice of shutter time, aperture and gain is exposure. Multi-frame methods such as burst denoising and merging belong to computational photography.

How It Works

Szeliski (2022, Section 2.3, Fig. 2.23) draws the camera as a chain: scene radiance passes through the optics and aperture, the shutter limits the exposure, the sensor converts light to charge, an amplifier applies the gain (the ISO setting), an analog-to-digital converter (ADC) produces the RAW values, and the ISP demosaics, denoises and sharpens, white-balances, applies a gamma or tone curve and compresses the result into a JPEG. Cameras differ most in how much of the ISP they run and whether they let the user skip it.

Optics

The lens’s focal length and the sensor size set the field of view, and the focal length in pixels enters the intrinsic matrix (camera parameters). Focusing makes one distance sharp, and the aperture sets the light and the depth of field (exposure); image formation derives the blur, diffraction and vignetting, and lens distortion the geometric deviations of real lenses. For a vision system, focus and zoom are part of the calibration: changing either changes the intrinsics and the distortion.

Shutter

A mechanical shutter blocks light physically in front of the sensor; an electronic shutter starts and ends integration by resetting and reading the pixels. Photographic cameras offer both and hybrids: as of October 2026 the online manual of Nikon’s Z6III offers a mechanical shutter, an electronic front-curtain shutter (which it says reduces blur from camera shake at slow shutter speeds, with 1/2000 s as the shortest exposure) and a fully electronic shutter (Nikon, Z6III Online Manual, Shutter Type).

How the exposure is timed across the image matters more for vision than how it is ended. GenICam’s naming convention for machine-vision cameras distinguishes three modes (SFNC 2.8, Section 4.7): global, in which all pixels are exposed for the same time at the same moment; rolling, in which groups of pixels, typically lines, are exposed for the same length of time one after another; and global reset, in which exposure starts for all pixels at once but ends line by line, so pixels are exposed for different lengths of time. The image sensor article explains rolling shutters and their effect on moving scenes. It follows from the definition of global reset (reasoning, not a statement of the standard) that it adds a brightness gradient down the image, relatively larger for short exposures.

Sensor

The sensor, usually behind a Bayer colour filter array, converts light to charge with shot noise, read noise and dark current added, and the ADC rounds the amplified charge to an integer above a black offset; the image sensor article covers it and its measurement under EMVA 1288. Two facts carry into the ISP: RAW values minus the black level are close to proportional to light below saturation, and their noise variance grows linearly with the signal.

The Image Signal Processor

The ISP is the processing that turns the RAW mosaic into a finished image. Delbracio et al. (2021, Section 3; section numbers here are those of the arXiv version) describe it as a dedicated chip in the camera system that runs the whole pipeline in milliseconds per image. Its exact steps are rarely published: Karaimer and Brown (2016) note that the pipeline is embedded in the camera’s hardware and may include manipulation unique to each manufacturer, and Delbracio et al. (2021) add that two manufacturers using the same ISP hardware can tune it to give different looks. Sources agree on the main stages but not on their exact order. Karaimer and Brown draw pre-processing (black-level offset, normalization, bad-pixel mask), white balance, demosaicing, post-processing (noise reduction and sharpening), colour transform and colour rendering (tone mapping, the sRGB gamma and colour manipulation), before display or compression (Karaimer & Brown, 2016, Fig. 1); Delbracio et al. (2021, Fig. 6) put demosaicing before white balance and noise reduction after tone mapping.

Black level, normalization and defects

The ADC output of a dark pixel is not zero: an offset keeps noise in dark pixels from being clipped at zero. The first step subtracts this black level and divides by the range up to the white level, the largest useful value. The DNG specification states the same model for RAW files: values pass through an optional linearization table, the black level is subtracted, the result is rescaled so that zero light is 0.0 and the maximum useful value is 1.0, and values above 1.0 are clipped; it recommends keeping negative values for the early stages, for better noise reduction in the shadows (Adobe, 2023, Chapter 5). Delbracio et al. (2021, Section 3.2) add that the black level often depends on ISO and gain, that factory-mapped defective pixels are interpolated from neighbours, and that a lens shading correction, a pre-calibrated gain mask that undoes the radial fall-off of light, is applied here and adjusted slightly per frame.

Demosaicing

Each photosite measured one colour; demosaicing interpolates the other two. Bilinear interpolation of each channel from its own samples is the simplest method (image formation shows its artifacts); better methods follow edges, such as the weighted interpolation that Delbracio et al. (2021, Fig. 8) illustrate. Delbracio et al. note that most demosaicing algorithms are proprietary and often also clip highlights, sharpen and begin denoising. Szeliski (2022, Section 10.1.1) describes demosaicing as a mostly linear but often non-stationary process. For a vision method two things follow: two-thirds of the values at each pixel are interpolated rather than measured, and the interpolation correlates the noise of neighbouring pixels.

White balance

The colour recorded by the camera is the product of the illuminant and the surface. White balance divides each channel by the camera’s response to the estimated illuminant, so that neutral surfaces come out neutral; Delbracio et al. (2021, Section 3.2) describe it as a diagonal scaling of the channels, with the illuminant taken from factory-calibrated presets or estimated from the image by an auto-white-balance algorithm. The gains are large: in the Darmstadt Noise Dataset’s metadata, which Brooks et al. (2019, Section 3.4) sample from, the red gain lies between 1.9 and 2.4 and the blue gain between 1.5 and 1.9. Szeliski (2022, Section 2.3.2) notes that some cameras use a general $3 \times 3$ “colour twist” instead of a diagonal. Because auto white balance is estimated per image, the same surface can receive different gains in consecutive frames.

Colour correction

After white balance the image is still in the camera’s own RGB, defined by its filters. A $3 \times 3$ colour correction matrix (CCM) maps it to a standard space (Brooks et al., 2019, Section 3.5): sRGB in their description, while Delbracio et al. (2021, Section 3.2) report that most cameras first convert to the wide-gamut ProPhoto RGB space for colour manipulation and later convert to an output space, most often sRGB, sometimes Adobe RGB or Display P3. The DNG specification stores, for one to three calibration illuminants, matrices that relate camera colour to CIE XYZ, interpolated according to the chosen white balance (Adobe, 2023, Chapter 6). Color spaces describes the targets.

A CCM usually has large diagonal entries and negative off-diagonal ones, because camera filters overlap more than the primaries of the target space. It follows (a short calculation, not a sourced statement) that the matrix amplifies noise: if the three input channels have independent noise of equal variance, the output channel $i$ has a variance larger by $\sum_j M_{ij}^2$, which for the first row of the matrix in the example below, $(1.60, -0.45, -0.15)$, is 2.8, a 67% increase in standard deviation. Colours outside the output gamut produce negative or above-one values, which are clipped.

Denoising and sharpening

Noise reduction trades noise against fine detail and is tuned to the ISO and exposure settings (Delbracio et al., 2021, Section 3.2). The sources differ on where it sits. Karaimer and Brown (2016) write that on a camera noise reduction is applied before the non-linear photo-finishing steps, so that research which denoises finished sRGB images, for lack of access to the pipeline, works under assumptions that do not match practice; Delbracio et al. (2021, Section 3.2) show it after colour manipulation and tone mapping and add that some ISPs denoise before photo-finishing, or both before and after. Sharpening, often combined with demosaicing, raises contrast at edges. Both change the spatial correlation of the noise, which is why EMVA 1288 Release 4.0 Linear, whose model needs white noise, cannot be applied once processing such as demosaicing, denoising or sharpening has taken place (EMVA 1288, Section 1.2).

Tone curve and gamma

The last colour steps make the image look right on a display. Gamma compression spends more output levels on dark tones, where vision is more sensitive (Brooks et al., 2019, Section 3.6), and an S-shaped tone curve, modelled on the characteristic curve of film, adds contrast (Section 3.7). Brooks et al. model the first as $x^{1/2.2}$ and the second as the smoothstep curve $3x^2 - 2x^3$, while noting that real cameras may use more complex, edge-aware local tone mapping. Delbracio et al. (2021, Section 3.2) describe the tone curve as a one-dimensional look-up table that also compresses 10–14-bit RAW values into 8 to 10 output bits, combined with colour manipulation in a three-dimensional look-up table that implements picture styles such as vivid or portrait. The output colour space’s own transfer function, such as the sRGB curve, is a final gamma encoding.

Compression

The last stage is usually compression. JPEG subsamples the chroma and quantizes the discrete cosine transform of $8 \times 8$ blocks; the quality setting mainly controls the quantization step, and stronger quantization adds block artifacts and “mosquito noise” (Szeliski, 2022, Section 2.3.3). Video codecs add motion-compensated prediction between frames, so errors depend on neighbouring frames as well (Szeliski, 2022, Section 2.3.3). Quantization zeroes small coefficients first, and pixel noise spreads over many small high-frequency coefficients, so it follows that compression also acts as a crude denoiser that changes the noise statistics; the example below measures this.

Data It Produces

RAW and processed output

A camera can deliver its data at two points of the chain. A RAW image stores the sensor values before most processing: a mosaic with one colour per pixel, 10 to 14 bits per value in the figures Delbracio et al. (2021, Section 3.2) give, linear in light after the black level is removed. The DNG specification describes raw files as unprocessed or minimally processed data whose white balance, tone mapping, noise reduction and sharpening can be chosen afterwards, but which must be processed before use; manufacturers’ raw formats are proprietary and vary between cameras, which DNG, a public extension of TIFF 6.0, addresses (Adobe, 2023, Chapters 1 and 2). A processed image, usually an 8-bit sRGB JPEG or a compressed video stream, is ready for display but has passed through every stage above.

Metadata

Images carry data about their capture. The Exif standard of the Camera & Imaging Products Association (CIPA), first established in 1995, records metadata such as the capture date and time and the shutter speed and is used in most consumer imaging devices, smartphones included; version 2.0 (1997) added GPS location, and version 3.0 (2023) added annotation descriptions, for example for AI training data, and an “original preservation image” kept for authenticity checks (CIPA, 2023). As of October 2026 CIPA lists Exif version 3.1, published on 30 January 2026, as the current edition (CIPA Standards list). Szeliski (2022, Section 2.3) notes that many of a sensor’s operating parameters can be read from Exif tags. For RAW processing, DNG adds tags that the pipeline needs, among them the black and white levels, the colour matrices and AsShotNeutral, the white balance selected at capture expressed as the camera’s response to a neutral colour (Adobe, 2023, Chapter 4). Its NoiseProfile tag stores a two-parameter model of the noise in linear RAW values, a standard deviation of $\sqrt{S x + O}$ for signal $x$, with $S$ for the signal-dependent shot noise and $O$ for the readout noise (Adobe, 2023, Chapter 4); this is the Poisson–Gaussian model of image noise, written into the file.

Machine-vision interfaces

Industrial and scientific cameras stream frames to a computer over standardized interfaces, among them GigE Vision, USB3 Vision, CoaXPress, Camera Link and Camera Link HS. GenICam, published by the EMVA, gives all of them one programming interface: each device describes its features in a self-describing XML file, the Standard Features Naming Convention (SFNC) fixes the names and meanings of features across vendors, the Pixel Format Naming Convention names pixel formats, and a transport-layer interface lets the same application enumerate devices and stream data whatever the cable (EMVA, GenICam Introduction).

The SFNC shows what such a camera exposes that a photographic camera usually hides. Frames can be triggered by software or by an electrical input line (SFNC 2.8, Section 5.6); pixel formats include raw Bayer formats such as BayerRG8 and BayerRG12 alongside monochrome and colour ones (Section 4.40); chunk data attaches per-frame metadata to the image payload, including a timestamp in nanoseconds and the exposure time actually used (SFNC 2.8, Sections 2.22 and 24); and the Precision Time Protocol (IEEE 1588) synchronizes the clocks of several cameras on a network (SFNC 2.8, Section 27.3). ISP-like stages appear as optional features: black level, gain, white-balance ratios, gamma, look-up tables and colour transformation matrices, several with explicit enable switches (SFNC 2.8, Sections 2.4–2.6). A vision engineer can therefore choose the processing.

Classes of Camera

Cameras differ less in principle than in which parts of the chain they emphasize.

• Consumer cameras such as DSLRs and mirrorless cameras have large sensors, adjustable apertures and wide zoom ranges (Delbracio et al., 2021, Fig. 4), a full photographic ISP, and usually a RAW option alongside JPEG. Delbracio et al. (2021, Section 2.2) give, as of 2021, pixels of roughly 4 µm and RAW values of 12 to 14 bits for a DSLR.
• Smartphone cameras trade optics for computation. Delbracio et al. (2021, Section 2), as of 2021, describe sensors of roughly 5 × 4 mm against 36 × 24 mm for a full-frame camera, compact lenses with fixed apertures, pixels of about 1.5 µm or less, and about 10 bits per RAW value; the thinness of the phone limits the focal length of each module, so phones carry two or more camera modules with different focal lengths and fields of view, from ultra-wide to telephoto, and some use folded optics. A phone has far more computing power than a DSLR, which it spends on merging several frames per photograph and on synthetic depth-of-field blur.
• Machine-vision cameras are built to be controlled and measured, with the shutter modes, triggers, pixel formats and metadata described above; their datasheets can report EMVA 1288 measurements.
• Scientific cameras push the sensor toward the physical limits. EMVA 1288 provides for cameras with internal temperature regulation and cooling of the sensor (EMVA 1288, Section 7.2), which reduces dark current for long exposures; their output is linear RAW data, often monochrome. Large astronomical cameras are mosaics of many sensors, as the image sensor article describes.

Calibration

Geometric calibration, of the intrinsics, distortion and pose from images of a known target, is described in camera calibration. Radiometric calibration measures the response function, the vignetting and the noise, with an integrating sphere, a uniformly lit target or a series of exposures (Szeliski, 2022, Sections 10.1.1–10.1.3); Szeliski also suggests photographing colour charts and comparing RAW and JPEG values to see what a camera does (Section 2.3.2). For processed images the response is difficult to model from first principles and must be measured (Szeliski, 2022, Section 10.1.1), and it can change with the camera’s settings and, through automatic tone mapping, with the scene. Systems that combine several cameras or other sensors also need the time of each exposure, which per-frame timestamps and clock synchronization provide, a time per row for a rolling shutter.

Processing the Data

Where processing breaks the assumptions of vision methods

Many vision methods model the pixel value as a measurement of light. The ISP violates that model in identifiable ways.

• Linearity. Gamma and tone curves make values a non-linear function of light. Ratios of intensities, shading, exposure compensation between frames and any method that adds or averages light (deblurring, photometric stereo, high dynamic range merging) need linear values; Szeliski (2022, Section 2.3.2) states that such computations must first undo the gamma and the per-image colour balance. Undoing a standard curve such as sRGB is not enough when the camera has also applied its own tone curve, as the example shows.
• Consistency between frames. Auto-exposure, auto white balance and scene-dependent tone mapping change the mapping from light to value from frame to frame. Brightness constancy, on which optical flow, direct visual odometry and stereo matching rely, is then violated even for a static scene under constant light.
• Noise model. In RAW data the noise is close to independent between pixels, with variance linear in the signal. Brooks et al. (2019, Section 3.1) contrast this with processed images, whose noise has complex characteristics because of non-linearities and correlation between pixel values; the stages above correlate, amplify, reshape and partly remove it. A denoiser or a noise-level estimate built for one domain is wrong in the other; image denoising discusses the consequences.
• Compression artifacts. Block artifacts and mosquito noise are structure that is not in the scene; it follows that they can create false corners and edges, alter fine texture and shift sub-pixel measurements.
• Geometry and flat field. ISPs can correct lens distortion and shading and resize or crop for digital zoom (Karaimer & Brown, 2016, Section 3; Delbracio et al., 2021, Section 3.2), which changes the effective intrinsics and the flat field; it follows that a calibration made in one camera mode need not hold in another.

Not every method suffers. Buckler, Jayasuriya and Sampson (2017) simulated pipelines with individual ISP stages removed, using a reversible pipeline model, and tested eight vision algorithms (CNN classifiers on CIFAR-10, Faster R-CNN on VOC 2007, face identification, Farneback optical flow and SGBM stereo on Middlebury, and structure from motion). They found colour transformations and gamut mapping had little effect, while demosaicing and gamma compression mattered: without gamma compression the structure-from-motion pipeline failed to find enough features, and without demosaicing the mean error of Farneback optical flow rose to about 0.45 in their Fig. 3a, from a baseline the figure shows at about 0.30 (the text gives 0.227 to 0.448). Only the stereo matcher was also sensitive to noise. They attribute the gamma result to the distribution of intensities: linear values follow a roughly log-normal distribution, which makes features harder to detect for both classical algorithms and CNNs. It follows that for detection and matching the gamma curve helps rather than hurts; note that these results come from simulated pipelines, inverted from processed images, rather than from cameras with stages switched off.

When to use RAW

It follows from these points (a summary, not a rule from one source) that RAW, or a machine-vision camera with its processing switched off, is the right input when a method measures light: photometry, colour measurement, high dynamic range merging, noise modelling and raw denoising, intensity comparisons across frames or cameras, and anything that depends on the noise statistics. Processed images are adequate, and sometimes better, for recognition and for feature-based geometry, provided the processing at test time matches the processing at training time; learned models inherit the pipeline of their training data. Where only processed images exist, the camera’s response can be calibrated and inverted, and the effects of its processing can be partly reversed by an approximate inverse pipeline, as Brooks et al. (2019) do for denoising.

Example: a Simple ISP and a Linear Measurement

The code renders six grey patches under a warm light, each reflecting exactly twice the light of the next, and records them with a simulated 12-bit RGGB sensor (shot noise, 4 electrons of read noise, black level 256 DN, gain 0.3 DN per electron) at one exposure time and at twice that time. A simple ISP subtracts the black level, demosaics bilinearly, white-balances with the known illuminant, applies a colour matrix whose rows sum to one (so greys stay grey), the gamma $x^{1/2.2}$ and the smoothstep tone curve of Brooks et al. (2019), quantizes to 8 bits and saves a JPEG at quality 75 with Pillow. In the green channel it measures two ratios that should both be 2, between adjacent patches and between the two exposure times, on RAW, on the JPEG codes, with only the gamma undone and with the full curve inverted, and reports each patch’s noise.

import io
import numpy as np
from scipy import ndimage
from PIL import Image

# Scene: six grey patches, each half as reflective as the last, 48 x 48 px.
refl = 0.9 * 0.5 ** np.arange(6)
P, H = 48, 48
W = P * len(refl)
scene = np.repeat(refl, P)[None, :].repeat(H, axis=0)        # 48 x 288
illum = np.array([0.55, 1.0, 0.7])   # warm light as seen by camera R, G, B

# Sensor: RGGB Bayer mosaic, shot and read noise, 12-bit ADC with offset.
black, white, K = 256, 4095, 0.3     # black level (DN), clip (DN), DN per e-
cfa = np.zeros((H, W), int)
cfa[0::2, 1::2] = 1; cfa[1::2, 0::2] = 1; cfa[1::2, 1::2] = 2

def capture(rng, electrons_white):
    mean_e = electrons_white * scene * illum[cfa]            # one colour/site
    e = rng.poisson(mean_e) + 4.0 * rng.standard_normal(mean_e.shape)
    return np.clip(np.round(K * e + black), 0, white)        # raw DN

def demosaic(lin):                   # bilinear, each channel from its sites
    kr = np.array([[1, 2, 1], [2, 4, 2], [1, 2, 1]]) / 4.0
    kg = np.array([[0, 1, 0], [1, 4, 1], [0, 1, 0]]) / 4.0
    return np.stack([ndimage.convolve(lin * (cfa == c), k, mode="mirror")
                     for c, k in [(0, kr), (1, kg), (2, kr)]], -1)

wb = illum[1] / illum              # white-balance gains (known illuminant)
ccm = np.array([[1.60, -0.45, -0.15],   # camera RGB -> linear sRGB;
                [-0.25, 1.45, -0.20],   # rows sum to 1, so greys stay grey
                [0.00, -0.50, 1.50]])

def isp(raw, quality=75):
    lin = (raw - black) / (white - black)                # black level, 0..1
    rgb = demosaic(lin) * wb                             # demosaic, WB
    rgb = np.clip(rgb @ ccm.T, 0, 1)                     # colour matrix
    g = rgb ** (1 / 2.2)                                 # gamma
    t = 3 * g**2 - 2 * g**3                              # tone curve
    img8 = np.round(255 * t).astype(np.uint8)            # 8-bit
    if quality is None:
        return img8.astype(float)
    buf = io.BytesIO()
    Image.fromarray(img8).save(buf, "JPEG", quality=quality)
    return np.asarray(Image.open(buf)).astype(float)     # decoded JPEG

def patch_stats(img):                # mean, variance in patch interiors
    out = []
    for i in range(len(refl)):
        p = img[8:-8, i * P + 8:(i + 1) * P - 8]
        out.append((p.mean(), p.var()))
    return np.array(out)

def measure(seed):
    rng = np.random.default_rng(seed)
    raw1, raw2 = capture(rng, 6000), capture(rng, 12000)   # t and 2t
    g = cfa == 1                                            # green sites
    def raw_green(raw):              # black-subtracted green samples
        return [raw[8:-8, i * P + 8:(i + 1) * P - 8]
                [g[8:-8, i * P + 8:(i + 1) * P - 8]] - black
                for i in range(len(refl))]
    a, b = raw_green(raw1), raw_green(raw2)
    j1, j2 = isp(raw1)[..., 1], isp(raw2)[..., 1]           # JPEG green
    gamma_only = lambda v: (v / 255) ** 2.2                 # undo gamma only
    def full_inverse(v):             # undo tone curve, then gamma
        y = np.clip(v / 255, 0, 1)
        return (0.5 - np.sin(np.arcsin(1 - 2 * y) / 3)) ** 2.2
    return dict(raw=np.array([p.mean() for p in a]),
                raw2=np.array([p.mean() for p in b]),
                var_mean=np.array([p.var() / p.mean() for p in a]),
                code=patch_stats(j1)[:, 0], sd=np.sqrt(patch_stats(j1)[:, 1]),
                sd8=np.sqrt(patch_stats(isp(raw1, None)[..., 1])[:, 1]),
                gam=patch_stats(gamma_only(j1))[:, 0],
                gam2=patch_stats(gamma_only(j2))[:, 0],
                inv=patch_stats(full_inverse(j1))[:, 0],
                inv2=patch_stats(full_inverse(j2))[:, 0])

m = measure(0)
adj = lambda v: v[:-1] / v[1:]       # each patch over the next-darker one
print("Ratio of adjacent patches (true 2):")
print("pair     raw  JPEG codes  gamma undone  full inverse")
for i, row in enumerate(zip(adj(m["raw"]), adj(m["code"]),
                            adj(m["gam"]), adj(m["inv"]))):
    print(f"{i}/{i + 1}  {row[0]:6.3f} {row[1]:11.3f} {row[2]:13.3f}"
          f" {row[3]:13.3f}")

print("\nSame patch at twice the exposure time (true 2), and noise:")
print("patch  refl    raw  gamma undone  full inverse | raw var/mean"
      "  sd 8-bit  sd JPEG")
for i in range(len(refl)):
    print(f"{i:5d} {refl[i]:5.3f} {m['raw2'][i] / m['raw'][i]:6.3f}"
          f" {m['gam2'][i] / m['gam'][i]:13.3f}"
          f" {m['inv2'][i] / m['inv'][i]:13.3f} |"
          f" {m['var_mean'][i]:12.3f} {m['sd8'][i]:9.2f} {m['sd'][i]:8.2f}")

print("\nAdjacent-patch ratios over seeds 0-9:")
runs = [measure(s) for s in range(10)]
for key in ["raw", "gam", "inv"]:
    r = np.array([x[key][:-1] / x[key][1:] for x in runs])
    print(f"{key:4s} min {r.min():.3f}  max {r.max():.3f}")

Output (NumPy 2.3.5, SciPy 1.16.3, Pillow 12.0.0):

Ratio of adjacent patches (true 2):
pair     raw  JPEG codes  gamma undone  full inverse
0/1   1.997       1.538         2.578         2.002
1/2   2.002       1.664         3.065         2.009
2/3   2.008       1.741         3.384         2.014
3/4   1.998       1.772         3.519         1.994
4/5   1.979       1.757         3.457         1.937

Same patch at twice the exposure time (true 2), and noise:
patch  refl    raw  gamma undone  full inverse | raw var/mean  sd 8-bit  sd JPEG
    0 0.900  1.998         1.803         2.000 |        0.291      1.67     0.92
    1 0.450  1.996         2.574         2.000 |        0.305      2.04     1.08
    2 0.225  1.996         3.058         2.006 |        0.311      1.89     1.02
    3 0.113  2.005         3.365         2.007 |        0.292      1.59     0.84
    4 0.056  2.005         3.595         2.018 |        0.319      1.43     0.76
    5 0.028  1.978         3.450         1.935 |        0.344      1.20     0.44

Adjacent-patch ratios over seeds 0-9:
raw  min 1.979  max 2.014
gam  min 2.567  max 3.770
inv  min 1.937  max 2.029

RAW values, once the black level is subtracted, give the right answer to within noise, in the shown run and over ten seeds.

The JPEG codes are not proportional to light, and the error depends on brightness: a doubling of light becomes a ratio between 1.54 and 1.77 in the shown run. Undoing only the standard gamma, as a method that linearizes the output with a fixed curve would, overshoots in the other direction, with ratios from 2.57 to 3.77 over the seeds, because the camera’s tone curve compressed the shadows and highlights and steepened the mid-tones. The same distortion appears between the two exposures, so compensating an exposure change by one global gain fails: each grey level would need its own. Inverting the full, known curve restores the ratios to within 1% for the brighter pairs, and to between 1.937 and 2.029 over the seeds; the lowest value in the shown run is the darkest pair, where an 8-bit code step is a large fraction of the signal and averaging noisy codes through a curved function biases the mean. Real cameras do not publish the curve, which is why the response must be calibrated or RAW used.

The noise columns show the noise model breaking. In RAW the variance divided by the mean stays close to the gain of 0.3, the linear photon-transfer relation of the sensor, and is highest in the darkest patch, where read noise adds a constant to the variance. After the ISP, the standard deviation in 8-bit codes rises with brightness and falls again for the brightest patch, shaped by the tone curve rather than by the sensor, and JPEG compression then roughly halves it, and cuts it by nearly two-thirds in the darkest patch. A noise model calibrated on RAW, or one of constant size, fits this output at no brightness.

Strengths and Limitations

A camera is passive, cheap and dense: millions of measurements per frame, in colour, at video rates. Its geometry is well modelled, and its RAW output is close to a linear measurement of light with a noise model that standards and file formats record. Its weakness for measurement is that what usually reaches a vision system is a rendering of that measurement, made by undocumented, scene-dependent processing. The sensor’s own limits, dynamic range, low-light noise, motion blur and rolling-shutter distortion, remain whatever the processing; exposure discusses the trade-offs, and no processing restores clipped detail.

Typical Applications

Cameras are the input of almost every vision task, from recognition and tracking to stereo vision, structure from motion and SLAM. Models trained on photographs from phones and consumer cameras inherit those cameras’ processing; machine-vision cameras serve inspection, robotics and metrology, where control of triggering, exposure and processing matters more than appearance; scientific cameras serve fields such as astronomy, where long exposures and linear data matter most.

Comparison with Other Sensors

A conventional camera samples intensity on a grid at fixed times and measures no distance. Event cameras instead report per-pixel brightness changes asynchronously rather than whole frames. Active sensors such as LiDAR and RGB-D cameras emit their own light and measure distance directly, at the cost of sparser or lower-resolution data than a camera’s. Cameras are often combined with such sensors, which makes the camera’s timing, calibration and processing part of a larger system.