Color Spaces

A color space is a rule for describing colors with numbers, usually three. The same pixel can be written as sRGB values, CIE XYZ tristimulus values, CIELAB coordinates, or YCbCr luma and chroma, and the choice determines what arithmetic on those numbers means. Most images on disk are sRGB, most video is YCbCr, and most colorimetric measurement is anchored to CIE XYZ; much vision code works in whichever space the image arrived in without noticing the difference.

Definition

A color space is a coordinate system for colors, together with a precise definition of what the coordinates mean. A device-independent color space is defined relative to human color matching (the CIE colorimetric observer), so its numbers denote the same color on any device. A device-dependent space, such as the raw RGB of a particular camera or display, gives numbers whose appearance depends on the hardware. Converting between spaces is a fixed mathematical mapping when both are fully specified.

Intuition

Human color vision is trichromatic: the retina has three types of cone, sensitive to long, medium, and short wavelengths, and any light, whatever its spectrum, is reduced to three cone responses. Two lights with different spectra but the same three responses look identical (they are metamers). Three numbers are therefore enough to specify a color as seen by a human, and different color spaces are different choices of those three numbers: amounts of three primary lights (RGB), a luminance plus two color differences (YCbCr), lightness plus two opponent axes (CIELAB), or hue, saturation, and value (HSV).

A camera has three color filters of its own, whose sensitivities do not exactly match the cones, so its raw RGB values are device-dependent. The camera pipeline converts them to a standard space, usually sRGB, before saving a JPEG.

Formal Definitions

CIE XYZ

The CIE 1931 standard colorimetric observer defines three color-matching functions, and the tristimulus values $X, Y, Z$ of a light are the integrals of its spectrum against them. $Y$ is defined to be proportional to luminance. CIE XYZ is the reference space through which other device-independent spaces are defined; its chromaticity coordinates $x = X/(X+Y+Z)$ and $y = Y/(X+Y+Z)$ give the familiar horseshoe-shaped chromaticity diagram. The CIE’s colorimetry report (CIE 015) is the standard reference for these definitions.

sRGB

sRGB, standardized as IEC 61966-2-1, is the default color space of the web and of most consumer images. It specifies three primaries with chromaticities red $(0.640, 0.330)$, green $(0.300, 0.600)$, and blue $(0.150, 0.060)$, the D65 white point $(0.3127, 0.3290)$, and a nonlinear transfer function. The primaries and white point are the same as those of HDTV in ITU-R BT.709.

Stored sRGB values $C'$ in $[0, 1]$ are gamma-encoded. They relate to linear light $C$ by

$$C = \begin{cases} C'/12.92, & C' \le 0.04045 \\[4pt] \left(\dfrac{C' + 0.055}{1.055}\right)^{2.4}, & C' > 0.04045 \end{cases}$$

and the inverse encoding is $C' = 12.92\,C$ for $C \le 0.0031308$ and $C' = 1.055\,C^{1/2.4} - 0.055$ otherwise. The curve approximates a pure power law with exponent about 2.2. Gamma encoding spends more of the 256 available 8-bit codes on dark tones, where vision is more sensitive to differences, which is why 8 bits suffice for display. The price is that encoded values are not proportional to light.

Linear sRGB maps to XYZ by a fixed $3 \times 3$ matrix. Its middle row gives relative luminance:

$$Y = 0.2126\,R + 0.7152\,G + 0.0722\,B, \qquad R, G, B \text{ linear}.$$

CIELAB and CIELUV

XYZ is not perceptually uniform: equal distances in it do not correspond to equal visible differences. In 1976 the CIE defined two more uniform spaces, CIELAB ($L^*a^*b^*$) and CIELUV ($L^*u^*v^*$). Both use a lightness $L^*$ computed from a cube-root-like function of $Y$ relative to a reference white, and two chromatic axes. In CIELAB, $a^*$ runs from green to red and $b^*$ from blue to yellow. The color difference $\Delta E^*_{ab}$ is the Euclidean distance between two CIELAB colors. Its uniformity is only approximate, so later formulas such as CIEDE2000 correct it; they are used in color quality control and in evaluating color reproduction.

YCbCr

Video and JPEG separate an image into luma $Y'$ and two color-difference (chroma) signals, $C_b \propto B' - Y'$ and $C_r \propto R' - Y'$. Luma is a weighted sum of gamma-encoded values (the prime marks encoding), so it is not the same as luminance $Y$. The weights depend on the standard. Standard-definition video (ITU-R BT.601) uses

$$Y' = 0.299\,R' + 0.587\,G' + 0.114\,B',$$

and HDTV (ITU-R BT.709) uses

$$Y' = 0.2126\,R' + 0.7152\,G' + 0.0722\,B'.$$

Studio video in both standards stores 8-bit luma in the range 16 to 235 and chroma in 16 to 240, leaving headroom and footroom. The JPEG File Interchange Format (JFIF) instead uses the BT.601 weights scaled to the full 8-bit range, 0 to 255, with chroma offset by 128. The separation lets codecs store chroma at lower resolution (chroma subsampling, such as 4:2:0), because vision is less sensitive to fine detail in color than in brightness.

HSV and HSL

HSV (hue, saturation, value) and HSL (hue, saturation, lightness) are cylindrical rearrangements of RGB. Alvy Ray Smith presented HSV and related transform pairs in 1978 as models of the monitor gamut organized by intuitive variables, for use in paint programs. Hue is an angle around the gray axis, saturation the distance from gray, and value is $\max(R, G, B)$. They are cheap to compute and convenient for picking colors or thresholding by hue, but they are not perceptually uniform: pure blue and pure yellow have the same value of 1 while differing greatly in perceived lightness, and hue is undefined and unstable for near-gray pixels.

Why the Choice Matters in Vision

Separating brightness from chromaticity gives some robustness to illumination. Shading and shadows change intensity much more than chromaticity, so normalized chromaticity, hue, or the chroma channels of YCbCr or CIELAB are used for color-based segmentation, tracking, and skin detection. The robustness is partial: a change in the illuminant’s color shifts all chromaticities, which is the problem of color constancy and the reason cameras perform white balance.

Perceptually uniform spaces suit tasks where distances should match visible differences, such as clustering colors for segmentation or measuring color error. YCbCr is the natural space for working with compressed images and video, and many algorithms operate on the luma channel alone, treating it as a grayscale image.

Learned models usually take the gamma-encoded sRGB values they were trained on, normalized per channel. They can learn whatever transformation helps, but inputs at test time must be in the same color space and channel order as in training. Physically based work, such as photometric stereo, radiometric calibration, and image formation models, needs linear values, so it uses raw sensor data or undoes the sRGB transfer function first.

Example

The example converts sRGB to linear light, compares luma with luminance, averages two pixels both ways, and shows that HSV value ignores perceived lightness.

import colorsys
import numpy as np

def srgb_to_linear(c):
    c = np.asarray(c, dtype=np.float64)
    return np.where(c <= 0.04045, c / 12.92, ((c + 0.055) / 1.055) ** 2.4)

def linear_to_srgb(c):
    c = np.asarray(c, dtype=np.float64)
    return np.where(c <= 0.0031308, 12.92 * c, 1.055 * c ** (1 / 2.4) - 0.055)

# Average a black and a white pixel (8-bit sRGB values)
black, white = np.array([0, 0, 0]), np.array([255, 255, 255])
naive = (black + white) / 2
lin = (srgb_to_linear(black / 255) + srgb_to_linear(white / 255)) / 2
correct = linear_to_srgb(lin) * 255
print("average of encoded values:", naive)
print("average in linear light:  ", correct.round(1))

# Luma (BT.709 weights on encoded values) vs relative luminance (linear)
pixels = np.array([[255, 0, 0], [0, 255, 0], [0, 0, 255], [128, 128, 128]]) / 255
w = np.array([0.2126, 0.7152, 0.0722])
print("luma Y':     ", (pixels @ w).round(3))
print("luminance Y: ", (srgb_to_linear(pixels) @ w).round(3))

# HSV: pure blue and pure yellow have the same V and S, very different luminance
for name, rgb in [("blue", (0, 0, 1)), ("yellow", (1, 1, 0))]:
    h, s, v = colorsys.rgb_to_hsv(*rgb)
    Y = srgb_to_linear(np.array(rgb, dtype=float)) @ w
    print(f"{name:6s} H={h * 360:5.1f} S={s:.1f} V={v:.1f} luminance={Y:.3f}")

Output:

average of encoded values: [127.5 127.5 127.5]
average in linear light:   [187.5 187.5 187.5]
luma Y':      [0.213 0.715 0.072 0.502]
luminance Y:  [0.213 0.715 0.072 0.216]
blue   H=240.0 S=1.0 V=1.0 luminance=0.072
yellow H= 60.0 S=1.0 V=1.0 luminance=0.928

A 50/50 mix of black and white light is encoded as about 188, not 128. Mid-gray 128 has luma 0.502 but emits only about 22% of white’s luminance. Luma and luminance agree for the pure primaries only because their encoded values are 0 or 1.

Common Misconceptions and Pitfalls

• Doing linear math on encoded values. Averaging, blurring, resizing, and blending sRGB values directly darkens edges and transitions, because the values are not proportional to light. The error is often tolerated for speed, but physically meaningful operations should run in linear light.
• Confusing luma with luminance. $Y'$ from encoded values and $Y$ from linear values differ, and the weights differ between BT.601 and BT.709. Grayscale conversions in different libraries use different conventions.
• BGR versus RGB. OpenCV loads color images in BGR order, while most other libraries and pretrained models use RGB. Feeding one to code expecting the other swaps red and blue without any error.
• Assuming RGB is one space. Camera raw RGB, sRGB, Display P3, and Adobe RGB have different primaries; the same triple denotes different colors in each.
• Ignoring out-of-gamut values. Conversions between spaces can produce negative or greater-than-one values for colors the target space cannot represent. Clipping them changes hue and saturation; for intermediate computations, keep floating point and clip only at the end.
• Treating HSV distances as perceptual. Hue is circular (0° and 360° are the same), and HSV distances do not reflect visible differences.

Where It Is Used

Color spaces appear wherever images are captured, stored, or compared: camera pipelines convert sensor values to sRGB; codecs work in YCbCr; color-based segmentation and tracking use chromaticity, hue, or CIELAB; and color constancy methods estimate the illuminant to correct colors. The distribution of values in each channel is summarized by the image histogram, and noise statistics differ between linear raw data and encoded images, as discussed in image noise. Downsampling color images should respect both sampling and aliasing and linearity.