Multispectral Satellite Imaging

A multispectral satellite imager is a passive optical camera in orbit that measures the sunlight reflected from the Earth in a handful of separate wavelength bands, four on Landsat 1’s scanner and 13 on Sentinel-2, from the visible to the shortwave infrared, and sometimes adds thermal bands that measure emitted heat. Its pixels are calibrated measurements rather than picture values. Each pixel of each band can be converted to the radiance that reached the instrument and then to a reflectance, the fraction of incoming sunlight that the surface sent back. That is what allows images from different dates, places and satellites to be compared, and what turns simple band arithmetic such as the vegetation index into a physical measurement.

Remote sensing describes the field in outline, including processing levels, NDVI, mixed pixels and the missions. This article goes inside the instrument, from the scanner and its bands to calibration, registration and the indices computed from the calibrated bands. Radar is the subject of synthetic aperture radar.

How It Works

What the instrument measures

Sunlight crosses the atmosphere, is reflected by the surface, and crosses the atmosphere again before it enters the telescope. The quantity at the entrance is spectral radiance, power per unit area, solid angle and wavelength, which ESA gives in W m⁻² sr⁻¹ µm⁻¹ for Sentinel-2. The detectors, whose physics the image sensor article describes, turn the light in each band into a charge, and the electronics turn the charge into an integer, the digital number or count. Everything that follows in processing is about undoing this chain: the instrument’s gain and offset give radiance, the sun’s irradiance and angle give top-of-atmosphere reflectance, and a model of the atmosphere gives surface reflectance.

The instrument splits the light into bands in one of a few ways. On Sentinel-2’s Multi-Spectral Instrument (MSI), a three-mirror telescope with a pupil equivalent to 150 mm sends the light through a beam splitter onto two focal planes, one for the ten visible and near-infrared (VNIR) bands and one for the three shortwave infrared (SWIR) bands. Stripe filters mounted over the detectors separate the individual bands. The VNIR detectors are CMOS, and the SWIR detectors are mercury cadmium telluride hybridized on a CMOS readout and kept passively below 195 K (ESA, S2 Mission). Landsat 8’s Operational Land Imager (OLI) uses the same principle: each band has its own filter, and the filters sit over the detector arrays like a butcher block, with silicon PIN detectors for the visible and near infrared and mercury cadmium telluride for the shortwave infrared (NASA, OLI).

Whiskbroom and pushbroom scanners

A satellite camera does not take snapshots. It builds the image line by line as the platform moves, and the way it covers each line divides scanners into two families (Natural Resources Canada, Multispectral Scanning).

• Across-track or whiskbroom scanners sweep a mirror from one side of the swath to the other. A small bank of detectors, one or a few per band, sees one ground cell at a time, and successive sweeps, laid down as the platform advances, build up the image. Landsat 1’s Multispectral Scanner System used an oscillating mirror (NASA, Landsat 1), and the Thematic Mappers on Landsats 4 and 5 and the Enhanced Thematic Mapper Plus (ETM+) on Landsat 7 kept the design. NASA describes ETM+ as a whisk-broom instrument whose scan mirror swept back and forth perpendicular to the orbit, building the 185 km swath strip by strip. A Scan Line Corrector, a pair of mirrors behind the main optics, compensated for the satellite’s forward motion during each sweep, so that the scan lines came out straight instead of in a zigzag (NASA, ETM+).
• Along-track or pushbroom scanners have no scan mirror. A long linear array of detectors in the focal plane covers the whole swath at once, and the platform’s motion pushes it along the ground like the bristles of a broom; each band needs its own array (Natural Resources Canada). OLI has more than 7,000 detectors per band across its focal plane. NASA notes that it inherited the approach from the Advanced Land Imager demonstrated on the experimental EO-1 satellite in 2000 (NASA, OLI). Sentinel-2’s MSI is pushbroom too. Its focal planes each carry 12 detector modules, staggered in two rows to cover the 20.6° field of view, which gives a 290 km swath (ESA, S2 Mission).

The tutorial of Natural Resources Canada gives the trade-off. The time that a detector looks at one ground cell, the dwell time, is short in a mirror scanner, and this constrains its spatial, spectral and radiometric resolution together. In a pushbroom each detector looks at its cell for longer, so it collects more energy. That improves radiometric resolution, or allows smaller pixels and narrower bands at the same radiometric resolution. Solid-state arrays also have no moving parts. Their cost is that thousands of detectors must be cross-calibrated to respond uniformly, which the tutorial calls necessary and complicated. Mirror scanners have a geometric weakness of their own: the distance to the ground grows towards the edges of the swath, so the ground cells grow and the image is distorted (Natural Resources Canada).

A rough calculation shows how large the difference in dwell time can be. Sentinel-2 orbits in 100.6 minutes (ESA, S2 Mission), so its ground track moves at about 6.6 km/s, ignoring the Earth’s rotation, and a 10 m line passes in about 1.5 ms. A pushbroom detector can integrate for that whole time. A hypothetical whiskbroom with one detector per band, sweeping the same 290 km swath at 10 m, would have to visit 29,000 cells in the same 1.5 ms, about 50 ns each (our calculation; real whiskbrooms used several detectors per band to relax this, as NASA’s description of earlier Landsat instruments sending light “to a few detectors” indicates). NASA attributes OLI’s better signal-to-noise ratios partly to the pushbroom architecture, which also removes the mechanical noise of moving parts, and partly to narrower bands placed away from atmospheric absorption, onboard calibration and finer quantization. It gives OLI signal-to-noise ratios of 100 to 300 at 5% of maximum radiance, against 15 to 50 for ETM+ (NASA, OLI).

Pushbroom focal planes also explain an effect that matters for vision. On Sentinel-2 the detectors for the different bands sit at different positions along the track, so each band sees a given point at a slightly different time. ESA tabulates the delays: 0.264 s from B02 (blue) to B08 (near infrared), 1.005 s from B02 to B04 (red), and up to 2.586 s from B02 to B09. The staggered modules add a parallax of up to about 46 km along track between odd and even detectors, and the band layout an inter-band parallax of up to about 14 km (ESA, S2 Mission). Processing registers the bands for stationary ground, but anything that moves is imaged at a different place in each band. A ship at 10 m/s moves about 10 m, one pixel, between the blue and the red band, and an aircraft at 250 m/s about 250 m (our calculation from ESA’s delays, with illustrative speeds). Moving objects therefore appear displaced from band to band, as coloured fringes or separate copies (our reasoning). ESA’s Level-2A processor turns the parallax to use: since Sen2Cor 2.10 it estimates cloud-top height from it for the cloud-shadow mask and uses it to remove bright buildings falsely detected as cloud and snow detected in the sky (ESA, S2 Processing).

Spectral bands and response functions

A band is not a single wavelength. Each detector responds to a range of wavelengths with a sensitivity $S(\lambda)$, the band’s spectral response function, which falls off gradually at both edges. The signal of a band is therefore a weighted average of the spectral radiance over the response function. ESA characterizes each Sentinel-2 band by its equivalent wavelength, the barycentre of the response function, not weighted by the solar spectrum:

$$\lambda_c = \frac{\int \lambda \, S(\lambda) \, d\lambda}{\int S(\lambda) \, d\lambda},$$

and by its bandwidth (ESA, S2 Mission). The equivalent wavelengths and bandwidths differ slightly from one satellite to the next: for band 2 they are 492.7 nm and 65 nm on Sentinel-2A, 492.3 nm and 65 nm on Sentinel-2B, and 489.1 nm and 65 nm on Sentinel-2C. These details matter. ESA states that an error of 1 nm in the equivalent wavelength induces errors of several per cent in reflectance, especially in the blue and the near infrared, and that the equivalent wavelengths are known to better than 1 nm.

Band placement follows the physics of the surfaces and of the atmosphere. Sentinel-2’s 13 bands, as ESA groups them, are four 10 m bands (blue, green, red and a broad near-infrared band), six 20 m bands (four narrow bands in the red edge, where vegetation reflectance rises steeply, and two broader SWIR bands for snow, ice and cloud detection and vegetation moisture), and three 60 m bands for aerosols, water vapour and cirrus detection. Bands are also kept out of the atmosphere’s absorption features or deliberately placed in them. OLI’s near-infrared band 5 (0.845–0.885 µm) was narrowed to exclude a water vapour absorption feature that the ETM+ near-infrared band contained. Its new band 9 (1.360–1.390 µm) sits on a strong water vapour absorption on purpose. Little light from the surface reaches the sensor there, so high, thin cirrus clouds stand out against a dark ground (NASA, OLI; NASA, Spectral Bands and Applications).

Two consequences follow for anyone combining data from several sensors. A “near-infrared band” is not one quantity: Sentinel-2’s B8 is 114–118 nm wide, depending on the unit, and its B8a about 20 nm, while OLI’s band 5 spans 0.845–0.885 µm. And an index computed from two sensors’ red and near-infrared bands will differ somewhat even over the same surface, because each band averages the surface spectrum over a different response function (our reasoning from the band definitions). Harmonized products address this. ESA’s Sen2Like tool, for example, produces surface reflectance made radiometrically consistent across Sentinel-2 and other optical missions such as Landsat (ESA, S2 Mission).

Data It Produces

A multispectral product is a stack of co-registered images, one per band, with metadata. What the pixel values mean depends on the processing level, which Remote Sensing introduces for Sentinel-2: counts, top-of-atmosphere (TOA) radiance (Level-1B), TOA reflectance on map tiles (Level-1C), which removes the effect of the sun’s irradiance and angle but not of the atmosphere, and surface reflectance (Level-2A). The bands come at several resolutions: 10, 20 and 60 m on Sentinel-2, 30 m with a 15 m panchromatic band and 100 m thermal bands resampled to 30 m on Landsat 8 and 9 (NASA, TIRS). Stored values are scaled integers, with the quantification value and band offsets that Remote Sensing describes given in the product metadata (ESA, S2 Processing). Products also carry sun and viewing angles, quality and saturation masks and, at higher levels, cloud masks.

For a vision model, a multispectral image is therefore an array of physical quantities whose channels differ in noise, native resolution and meaning, and whose values for the same surface change with the processing level, as the example below shows.

Characteristics

The four resolutions describe what the instrument can see. ESA defines them for Sentinel-2 (ESA, S2 Mission); this section goes into what limits each.

Spatial resolution

ESA defines spatial resolution as the at-ground representation of an individual detector in the array. For a detector of angular size $\theta_{\mathrm{IFOV}}$, its instantaneous field of view, seen from an altitude $H$ at nadir, the ground cell is about $H \, \theta_{\mathrm{IFOV}}$ across, which is the small-angle form of the tutorial’s statement that the IFOV and the altitude determine the ground resolution cell (Natural Resources Canada). EnMAP’s specifications illustrate it: an IFOV of 9.5 arcseconds from 653 km gives 653 km × 9.5 / 206,265 ≈ 30 m, the specified ground sampling distance (our calculation from the EnMAP Specifications). Off nadir the cell grows, because the distance grows and the ground is viewed obliquely.

The ground cell is not the whole story. The optics, the detector’s own extent and the motion during integration blur the image, and the blur is described by the point spread function or, in the frequency domain, the modulation transfer function (MTF), the contrast that the system preserves at each spatial frequency. At the Nyquist frequency, the finest pattern the pixel grid can represent (see sampling and aliasing), Sentinel-2’s MTF requirement is between 0.15 and 0.3. ESA’s page states the requirement in two places that disagree: its list gives 0.15 to 0.3 for the 10 m bands and less than 0.45 for the 20 m and 60 m bands, while its text gives 0.15 to 0.3 for the 10 m and 20 m bands and less than 0.45 for the 60 m bands. ESA also publishes Gaussian point spread functions for each band of each unit, estimated from MTF measurements made in orbit (ESA, S2 Mission). An MTF well below one at Nyquist means that neighbouring pixels are correlated and edges are spread over more than one pixel. An MTF too close to one would let finer detail alias into the image (our reading of the requirement’s two bounds). For vision models this sets what a “10 m pixel” contains: part of the signal of a bright object spills into its neighbours.

Remote Sensing distinguishes spatial resolution from the ground sample distance (GSD), the spacing of pixel centres; NASA’s TIRS page uses that definition and gives the thermal bands a GSD of 100 m, resampled to 30 m in products (NASA, TIRS). A resampled product has more pixels but no more resolution.

Spectral resolution

Spectral resolution is the instrument’s ability to distinguish features in the electromagnetic spectrum (ESA, S2 Mission). It is set by the number, placement and width of the bands and by the shape of their response functions. Narrow bands isolate specific absorption features, such as the red edge of vegetation, but collect less light and so need a longer dwell time or larger pixels for the same signal-to-noise ratio; this is the pushbroom trade-off above in spectral form (Natural Resources Canada). Hyperspectral imagers, described below, push spectral resolution to contiguous bands a few nanometres wide.

Radiometric resolution

Radiometric resolution is how finely the instrument can distinguish differences in intensity. It is usually quoted in bits: 8 for ETM+, 12 for OLI on Landsat 8 and for Sentinel-2’s MSI, 14 for OLI-2 on Landsat 9 (NASA, Landsat 8; NASA, Landsat 9; ESA, S2 Mission). NASA states that Landsat 9’s move to 14 bits improved sensitivity by about 25% and benefits dark targets such as coastal waters (NASA, OLI). ESA adds that radiometric resolution also depends on the detectors’ signal-to-noise ratio. The bit depth is only useful while the quantization step is smaller than the noise, and the signal-to-noise ratio is the more informative figure. ESA lists it at a reference radiance for each band: 154 for B2, 142 for B4 and 174 for B8 on Sentinel-2A and 2B, for example (ESA, S2 Mission). If the 12 bits spanned B4’s sensitivity range from zero to its maximum radiance of 484 W m⁻² sr⁻¹ µm⁻¹ linearly, one count would be about 0.12 in those units, roughly 0.1% of the reference radiance of 108, while the noise at that radiance, at a signal-to-noise ratio of 142, is about 0.7% (our calculation under that idealized linear mapping). Noise rather than quantization would then limit the measurement at typical radiances, and the extra bits matter mostly for dark targets.

Temporal resolution

Temporal resolution is the revisit interval, set by the orbit, the swath and the number of satellites rather than by the camera; Remote Sensing gives the Landsat and Sentinel-2 values. Wide swaths buy frequent revisits: Sentinel-2’s 290 km swath compares with 185 km for Landsat 5 TM and Landsat 7 ETM+ and 120 km for SPOT-5, as ESA notes. EnMAP’s 30 km swath gives a nadir revisit of 27 days, shortened to 4 days by pointing up to 30° off nadir (EnMAP, Mission). Cloud cover makes the effective revisit of an optical sensor longer than the nominal one, which is why Sentinel-2A’s extension campaign adds acquisitions over tropical regions that are frequently cloudy (ESA, S2 Mission).

Calibration

From counts to reflectance

Radiometric calibration relates the counts to radiance. For Sentinel-2, ESA describes the Level-1B chain as a sequence of detector-level corrections. Dark signal, the output with no light, is subtracted, including a per-line offset estimated from blind pixels at the ends of each detector module. Crosstalk between bands is removed. Equalization corrects the non-linearity of each pixel’s response and its differences from the other pixels, so that a uniform landscape gives a uniform image; ESA models the response as a cubic function of radiance for VNIR pixels and as bilinear for SWIR pixels. Defective pixels are corrected, and the 60 m bands are binned (ESA, S2 Processing). Without equalization, a pushbroom image shows stripes along the track, because each column of the image comes from one detector (our reasoning from the column geometry of a pushbroom).

The equalized count is then converted to TOA reflectance. ESA gives the conversion as

$$\rho_k(i,j) = \frac{\pi \, \mathrm{CN}_{k}(i,j)}{A_{k} \, E_{s} \, d(t) \, \cos\theta_s(i,j)},$$

where $\mathrm{CN}_k$ is the equalized count of band $k$ after removing the Level-1B offset, $A_k$ is the absolute calibration coefficient, $E_s$ is the equivalent extra-terrestrial solar irradiance in the band, $d(t)$ is the correction for the varying distance between the Sun and the Earth, and $\theta_s$ is the sun zenith angle at the pixel. ESA’s equation also indexes $\mathrm{CN}$ and $A$ by the number of time-delay integration lines used (ESA, S2 Processing). Dividing the count by $A_k$ gives radiance. The factor $\pi / (E_s \cos\theta_s)$ then divides that radiance by the radiance a perfectly white, diffusely reflecting surface would send back under the same sun, which is why reflectance is dimensionless and comparable between dates with different sun angles (our reading of the equation). The solar irradiance that Sentinel-2 uses is the Thuillier 2003 spectrum integrated over each band’s response; ESA tabulates it, for example 1512.06 W m⁻² µm⁻¹ for B4 on Sentinel-2A and 1041.63 for B8, and gives alternative values from the newer TSIS model, which differ by up to about 5% in the SWIR.

Keeping the calibration right in orbit

Calibration coefficients drift, so they are measured again throughout the mission. Sentinel-2 acquires dark images over the ocean in the eclipse part of the orbit, generally twice a month. Once a month, over the North Pole, a shutter mechanism moves a diffuser into the beam and the instrument images sunlight reflected by it. These acquisitions update both the absolute and the relative gains; the diffuser’s BRDF was characterized before launch (ESA, S2 Processing). Two or three times a year, experts validate the calibration vicariously, over snow at Dome C in Antarctica, instrumented sites of the CEOS network such as La Crau, and desert sites used to cross-calibrate with other sensors. ESA’s Level-1 requirements are an absolute radiometric uncertainty better than 5% (goal 3%), an inter-band relative uncertainty of 3% and a multi-temporal one of 1% (ESA, S2 Mission).

Landsat follows the same pattern with different hardware. OLI carries a shutter, a solar diffuser that reflects sunlight into the telescope, and two assemblies of six lamps each inside an integrating hemisphere (NASA, OLI), and Landsat 8 turns to scan the Moon at every full moon (NASA, Landsat 8). OLI’s requirement is a calibration uncertainty of less than 5% in radiance and less than 3% in TOA reflectance (NASA, OLI). NASA and the USGS track calibration with onboard sources and verify it independently over pseudo-invariant calibration sites, stable surfaces such as White Sands in New Mexico and Railroad Valley Playa and Lake Tahoe in Nevada, and in field campaigns. When systematic shifts appear, the Calibration Parameter File is updated (NASA, Landsat Calibration & Validation). The thermal bands are calibrated differently. TIRS subtracts its own thermal emission by comparing the view of the Earth with a view of cold space, converts the signal to radiance with functions measured before launch, and checks them against an onboard blackbody (NASA, TIRS).

Geometric calibration and band-to-band registration

Geometric calibration ties each pixel to a ground position, and band-to-band registration makes the bands of one image line up with each other. They are separate requirements. For Sentinel-2, ESA specifies an absolute geolocation uncertainty of 20 m (2σ) without ground control points, a multi-temporal co-registration of 0.3 pixels or better (2σ), and, between any two bands, a co-registration better than 0.30 of the coarser of the two bands’ sampling distances at 3σ (ESA, S2 Mission). The last figure is easily misread. Between two 10 m bands it allows 3 m, but between a 10 m band and a 20 m band it allows 6 m, which is 0.6 of a 10 m pixel.

Level-1C processing achieves this registration by modelling the geometry rather than by matching the bands’ content directly. For each band and each detector, it computes resampling grids that map the target orthoimage into the instrument’s native geometry, using the refined viewing model and a digital elevation model, and it interpolates the radiance at each target point with cubic splines (ESA, S2 Processing). The viewing model is refined by matching the reference band B04 against the Global Reference Image. ESA also describes a VNIR–SWIR co-registration step that is deactivated in operational processing. The resampling step is a choice with consequences: every value in a Level-1C product is an interpolated one, and a user who resamples the 20 m bands to 10 m to stack them with the others interpolates a second time (our reasoning). Multi-date registration, the requirement behind comparisons between dates, belongs to change detection, which describes its tolerances; the general methods are those of image registration.

Residual misregistration between bands matters because almost every use of multispectral data combines bands pixel by pixel. If the near-infrared band is offset from the red one, a pixel’s index combines the near infrared of one patch of ground with the red of another. The example below measures the effect.

Processing the Data

Atmospheric correction in outline

The atmosphere adds light scattered towards the sensor, the path radiance, and attenuates the light from the surface on both legs of its journey. NASA notes that short visible wavelengths are scattered by small atmospheric particles, which makes OLI’s coastal band useful for detecting aerosols, while shortwave infrared is less affected by such scattering (NASA, Spectral Bands and Applications); water vapour absorbs in specific bands. Atmospheric correction estimates the atmosphere’s state and inverts these effects to give surface reflectance.

ESA’s Level-2A processor illustrates the usual ingredients (ESA, S2 Processing). Look-up tables generated once with the libRadtran radiative transfer model hold, for a wide range of atmospheres, sun angles and ground elevations, the path radiance, the direct and diffuse transmittances and fluxes, and the spherical albedo, computed at 0.6 nm and resampled with Sentinel-2’s response functions. The aerosol optical thickness is estimated from the image itself, over reference areas of known behaviour, preferably dark dense vegetation and water. The visibility, and from it the aerosol optical thickness, is derived from a correlation of the reflectance in band 12 (2.19 µm) with that in the blue and red bands. If the scene has too few dark pixels, ESA’s processor falls back on aerosol estimates from the Copernicus Atmosphere Monitoring Service supplied with the product. Water vapour is retrieved by comparing band 9, in an absorption region, with band 8a, in an atmospheric window. The baseline aerosol type is rural/continental. Terrain effects and cirrus are corrected too, and ESA also supplies a separate processor, Sen2Water, for water-leaving reflectance over lakes, rivers and coastal waters (ESA, S2 Mission).

Two practical consequences follow. Indices and models applied to TOA reflectance carry the atmosphere’s contribution, which varies from scene to scene. And surface reflectance is an estimate whose error is largest where the surface is dark and the atmosphere hazy (our reasoning from the structure of the correction).

Spectral indices

Remote Sensing describes NDVI, the normalized difference of the near-infrared and red bands. Many other indices use the same form, the difference of two bands divided by their sum, with bands chosen for a different surface property. Montero et al. (2023) assembled a catalogue of spectral indices, Awesome Spectral Indices, with each index’s formula and source reference in a machine-readable form; its release 0.12.0 lists 280 indices across vegetation, water, burn, urban, soil, snow, cloud, radar and kernel domains. Some widely used ones, with formulas from that catalogue (G, R, N, S1 and S2 for the green, red, near-infrared, and first and second shortwave infrared bands, and B for blue):

Index	Formula	Target	Source given by the catalogue
NDWI	(G − N)/(G + N)	open water	McFeeters (1996)
MNDWI	(G − S1)/(G + S1)	open water	Xu (2006)
NDMI	(N − S1)/(N + S1)	vegetation moisture	Wilson and Sader (2002)
NBR	(N − S2)/(N + S2)	burned areas	USGS Open-File Report 02-11 (2002)
NDBI	(S1 − N)/(S1 + N)	built-up areas	Zha et al. (2003)
NDSI	(G − S1)/(G + S1)	snow	Riggs, Hall and Salomonson (1994)
EVI	2.5 (N − R)/(N + 6R − 7.5B + 1)	vegetation	Huete et al. (1997)

Water reflects more green than near infrared, unlike almost every land surface, so NDWI is positive over open water. The MNDWI replaces the near infrared with the shortwave infrared; Montero et al. describe the change as reducing, in particular, the noise produced by built-up areas. The normalized burn ratio comes from fire science. Key and Benson (2006) chose Landsat TM and ETM+ bands 4 (near infrared) and 7 (2.08–2.35 µm), the two bands that responded most, and in opposite ways, to burning in their experience in western U.S. forests: near-infrared reflectance falls and shortwave infrared rises after fire. They computed the ratio from reflectance corrected for atmospheric transmittance. The difference between the prefire and postfire ratios, dNBR, measures burn severity. They note that it assumes unburned ground is similar in phenology and moisture at the two dates and that the two images are adequately co-registered, and that burned areas can take strongly positive or negative values depending on whether the fire reduced or enhanced productivity.

The enhanced vegetation index is not a normalized difference. It adds the blue band and constants, which the catalogue gives as a gain of 2.5, coefficients of 6 and 7.5 for the aerosol resistance term and a canopy background adjustment of 1. Huete et al. (2002) evaluated the MODIS NDVI and EVI products over their first year; according to their abstract, NDVI saturated asymptotically in high-biomass regions such as the Amazon, while EVI remained sensitive to canopy variations.

Three cautions apply to all of these. First, names are not stable: the catalogue lists the normalized difference of near infrared and SWIR1 four times, as NDMI, as the Normalized Difference Infrared Index (NDII), as the Land Surface Water Index (LSWI) and as the Normalized Difference SWIR (NDSWIR, in the burn domain), each with its own source, and gives MNDWI and NDSI the same formula; an index should be reported with its formula and bands, not only its acronym. Second, the bands an index was defined on differ from the bands of today’s sensors; Key and Benson’s bands 4 and 7 correspond to different band numbers on OLI and Sentinel-2. Third, an index assumes a processing level. The example shows how far values move between counts, radiance, TOA reflectance and surface reflectance.

Using the bands in learned models

Montero et al. note that indices can be added to a dataset as features before machine learning, and Remote Sensing and land cover classification describe how networks are adapted to more than three bands. The sensor adds three requirements (our reasoning from the sections above): inputs at a known processing level with scale and offset removed; a deliberate choice of how bands at different resolutions are resampled to one grid; and awareness that misregistration, parallax and the MTF shape what appears at edges, where small objects lie.

Example: Indices at Each Processing Level and Misregistered Bands

The script works on a synthetic scene with known surface reflectances in six bands named after Sentinel-2’s B2, B3, B4, B8, B11 and B12 (blue, green, red, near infrared, SWIR1 and SWIR2). The six classes are water, dense vegetation, dry grass, bare soil, built-up land and a burn scar. The reflectances are illustrative values chosen for the example, not measured spectra.

The script has three parts. Part A computes seven indices for each class from surface reflectance. Part B passes the class spectra through the calibration chain in reverse and computes four of the indices at each level. A simplified atmosphere maps surface reflectance $\rho_s$ to TOA reflectance as $\rho_{\mathrm{TOA}} = \rho_{\mathrm{path}} + T \rho_s$. Its path reflectance falls from 0.08 in the blue to 0.002 in SWIR2, and its transmittance rises from 0.80 to 0.96. This is an illustrative linear model, not a radiative transfer calculation. TOA reflectance becomes radiance with ESA’s Sentinel-2A solar irradiances (Thuillier) and a sun zenith angle of 40°, and radiance becomes 12-bit counts with illustrative gains $A_k$. The counts are then converted back to TOA reflectance with ESA’s equation, with $d(t) = 1$. Part C builds a 1.6 km × 1.6 km scene on a 1 m grid: random fields of vegetation, dry grass and bare soil, a river 30 m wide, a town of 15 m blocks separated by 8 m strips of grass, and a burn scar. It images the scene at 10 m by averaging 10 × 10 blocks in each band, with the near-infrared band’s footprint shifted along $x$ by 0 to 1 pixel in steps of whole metres, so that no interpolation is involved. The aligned product is the reference. The scene is noise-free, so that every difference comes from misregistration. A pixel counts as an edge pixel when its 3 × 3 neighbourhood contains more than one majority class.

All parameters were fixed in a plan before the first run. Before that run, one error in the scene was corrected: the town’s gaps had the same class as its blocks, so the town had no edges, and they were changed to grass; a line that had no effect was also removed. The script below is the first run unchanged. It runs in under a second with NumPy 2.3.5, two runs gave identical output, and it takes the seed of the random scene as an argument (the output shown is seed 0).

import sys
import numpy as np

seed = int(sys.argv[1]) if len(sys.argv) > 1 else 0
rng = np.random.default_rng(seed)

# Illustrative surface reflectances (not measured spectra) in six bands named
# after Sentinel-2 bands: B2 blue, B3 green, B4 red, B8 NIR, B11 SWIR1, B12 SWIR2.
names = ["water", "vegetation", "dry grass", "bare soil", "built-up", "burned"]
R = np.array([[0.05, 0.06, 0.04, 0.02, 0.01, 0.005],
              [0.03, 0.07, 0.04, 0.40, 0.20, 0.09],
              [0.06, 0.09, 0.12, 0.25, 0.30, 0.20],
              [0.10, 0.14, 0.18, 0.25, 0.33, 0.28],
              [0.14, 0.15, 0.16, 0.20, 0.26, 0.24],
              [0.04, 0.05, 0.06, 0.10, 0.18, 0.20]])
B, G, RED, N, S1, S2 = range(6)

def nd(a, b):
    return (a - b) / (a + b)

def indices(x):
    return {"NDVI": nd(x[..., N], x[..., RED]),
            "EVI": 2.5 * (x[..., N] - x[..., RED])
                   / (x[..., N] + 6 * x[..., RED] - 7.5 * x[..., B] + 1),
            "NDWI": nd(x[..., G], x[..., N]),
            "MNDWI": nd(x[..., G], x[..., S1]),
            "NDMI": nd(x[..., N], x[..., S1]),
            "NBR": nd(x[..., N], x[..., S2]),
            "NDBI": nd(x[..., S1], x[..., N])}

print("A. Indices from surface reflectance")
ix = indices(R)
print(f"{'class':11s}" + "".join(f"{k:>7s}" for k in ix))
for i, n in enumerate(names):
    print(f"{n:11s}" + "".join(f"{v[i]:+7.2f}" for v in ix.values()))

# B. Calibration chain. Simplified, illustrative atmosphere: rho_toa = path + T * rho_s.
path = np.array([0.08, 0.05, 0.03, 0.015, 0.005, 0.002])
T = np.array([0.80, 0.85, 0.88, 0.92, 0.95, 0.96])
Es = np.array([1959.66, 1823.24, 1512.06, 1041.63, 245.59, 85.25])  # S2A, Thuillier (ESA)
A = np.array([10.0, 12.0, 15.0, 20.0, 60.0, 150.0])                 # illustrative gains
cos_t = np.cos(np.radians(40.0))
toa = path + T * R
L = toa * Es * cos_t / np.pi                     # radiance, W m-2 sr-1 um-1 (d = 1)
DN = np.clip(np.round(A * L), 0, 4095)           # 12-bit counts
toa_back = np.pi * DN / (A * Es * cos_t)         # ESA Equation 8 with d(t) = 1
print("\nB. The same indices computed at each processing level")
print(f"{'class':11s} {'index':5s} {'counts':>7s} {'radiance':>9s} {'TOA':>7s} {'TOA(DN)':>8s} {'surface':>8s}")
for i, n in enumerate(names):
    for k in ["NDVI", "EVI", "NBR", "NDBI"]:
        vals = [indices(x)[k][i] for x in (DN, L, toa, toa_back, R)]
        print(f"{n:11s} {k:5s} " + " ".join(f"{v:+7.3f}" if j != 1 else f"{v:+9.3f}"
                                            for j, v in enumerate(vals)))

# C. Band-to-band misregistration on a 1 m scene imaged at 10 m.
W, PAD = 1600, 10
WATER, VEG, GRASS, SOIL, BUILT, BURN = range(6)
cls = np.full((W, W + PAD), SOIL, dtype=np.int8)
for _ in range(60):                                  # fields
    y, x = rng.integers(0, W, 2)
    h, w = rng.integers(40, 300, 2)
    cls[y:y + h, x:x + w] = rng.choice([VEG, GRASS, SOIL], p=[0.5, 0.3, 0.2])
yy, xx = np.mgrid[0:W, 0:W + PAD]
cls[np.abs(yy - (800 + 150 * np.sin(xx / 180.0))) < 15] = WATER   # river 30 m wide
town = (yy >= 1100) & (yy < 1500) & (xx >= 100) & (xx < 600)
cls[town] = GRASS                                                  # 8 m green strips
cls[town & ((yy - 1100) % 23 < 15) & ((xx - 100) % 23 < 15)] = BUILT  # 15 m blocks
cls[200:420, 1100:1400] = BURN                                     # a burn scar
fine = R[cls][..., [G, RED, N]].astype(np.float64)                 # green, red, NIR

def box(a):                                     # 10 x 10 block mean -> 10 m pixels
    h, w = a.shape[0] // 10, a.shape[1] // 10
    return a[:h * 10, :w * 10].reshape(h, 10, w, 10).mean(axis=(1, 3))

g10, r10 = box(fine[:, :W, 0]), box(fine[:, :W, 1])
onehot = np.eye(6)[cls[:, :W]]
major = np.stack([box(onehot[..., c]) for c in range(6)], -1).argmax(-1)
pad = np.pad(major, 1, mode="edge")
nb = np.stack([pad[1 + dy:1 + dy + major.shape[0], 1 + dx:1 + dx + major.shape[1]]
               for dy in (-1, 0, 1) for dx in (-1, 0, 1)])
edge = (nb != major).any(axis=0)
print(f"\nC. NIR band offset along x; {major.size} pixels at 10 m, {100 * edge.mean():.1f}% at edges")
print("shift  mean|dNDVI| all/edge/inner   max   NDVI>0.4 flips  veg area  NDWI>0 flips  flips at edges")
ref = None
for k in [0, 1, 3, 6, 10]:
    n10 = box(fine[:, k:k + W, 2])
    ndvi, ndwi = nd(n10, r10), nd(g10, n10)
    if ref is None:
        ref = (ndvi, ndwi)
    d = np.abs(ndvi - ref[0])
    fv = (ndvi > 0.4) != (ref[0] > 0.4)
    fw = (ndwi > 0) != (ref[1] > 0)
    area = 100 * ((ndvi > 0.4).mean() - (ref[0] > 0.4).mean())
    flips = fv | fw
    at_edge = 100 * (flips & edge).sum() / max(flips.sum(), 1)
    print(f"{k / 10:4.1f} px   {d.mean():.4f} / {d[edge].mean():.4f} / {d[~edge].mean():.4f}"
          f"   {d.max():.3f}   {100 * fv.mean():5.2f}%        {area:+.2f} pp  {100 * fw.mean():5.2f}%"
          f"        {at_edge:5.1f}%")

A. Indices from surface reflectance
class         NDVI    EVI   NDWI  MNDWI   NDMI    NBR   NDBI
water        -0.33  -0.06  +0.50  +0.71  +0.33  +0.60  -0.33
vegetation   +0.82  +0.64  -0.70  -0.48  +0.33  +0.63  -0.33
dry grass    +0.35  +0.21  -0.47  -0.54  -0.09  +0.11  +0.09
bare soil    +0.16  +0.11  -0.28  -0.40  -0.14  -0.06  +0.14
built-up     +0.11  +0.09  -0.14  -0.27  -0.13  -0.09  +0.13
burned       +0.25  +0.09  -0.33  -0.57  -0.29  -0.33  +0.29

B. The same indices computed at each processing level
class       index  counts  radiance     TOA  TOA(DN)  surface
water       NDVI   -0.360    -0.478  -0.323  -0.322  -0.333
water       EVI    +0.244    +0.141  -0.152  -0.151  -0.056
water       NBR    +0.780    +0.967  +0.662  +0.665  +0.600
water       NDBI   -0.532    -0.814  -0.395  -0.396  -0.333
vegetation  NDVI   +0.687    +0.604  +0.709  +0.709  +0.818
vegetation  EVI   +10.279    -1.406  +0.799  +0.799  +0.636
vegetation  NBR    +0.752    +0.963  +0.625  +0.625  +0.633
vegetation  NDBI   -0.470    -0.786  -0.325  -0.325  -0.333
dry grass   NDVI   +0.248    +0.109  +0.287  +0.288  +0.351
dry grass   EVI    +1.071    -0.320  +0.249  +0.249  +0.214
dry grass   NBR    +0.346    +0.878  +0.116  +0.116  +0.111
dry grass   NDBI   -0.089    -0.564  +0.084  +0.084  +0.091
bare soil   NDVI   +0.089    -0.055  +0.131  +0.131  +0.163
bare soil   EVI    +0.288    +0.194  +0.120  +0.121  +0.111
bare soil   NBR    +0.192    +0.834  -0.050  -0.050  -0.057
bare soil   NDBI   -0.042    -0.531  +0.130  +0.130  +0.138
built-up    NDVI   +0.034    -0.109  +0.076  +0.076  +0.111
built-up    EVI    -0.844    +0.120  +0.090  +0.090  +0.090
built-up    NBR    +0.165    +0.826  -0.077  -0.078  -0.091
built-up    NDBI   -0.055    -0.540  +0.118  +0.117  +0.130
burned      NDVI   +0.086    -0.058  +0.128  +0.128  +0.250
burned      EVI    -0.299    +0.044  +0.079  +0.079  +0.086
burned      NBR    -0.053    +0.742  -0.289  -0.289  -0.333
burned      NDBI   +0.075    -0.441  +0.244  +0.243  +0.286

C. NIR band offset along x; 25600 pixels at 10 m, 27.5% at edges
shift  mean|dNDVI| all/edge/inner   max   NDVI>0.4 flips  veg area  NDWI>0 flips  flips at edges
 0.0 px   0.0000 / 0.0000 / 0.0000   0.000    0.00%        +0.00 pp   0.00%          0.0%
 0.1 px   0.0017 / 0.0062 / 0.0000   0.154    0.03%        -0.01 pp   0.08%        100.0%
 0.3 px   0.0051 / 0.0185 / 0.0000   0.405    0.33%        +0.02 pp   0.27%        100.0%
 0.6 px   0.0098 / 0.0358 / 0.0000   0.711    0.62%        +0.21 pp   0.48%        100.0%
 1.0 px   0.0154 / 0.0551 / 0.0003   0.975    1.23%        +0.76 pp   0.84%        100.0%

Each index separates a different pair of classes. With these spectra, NDVI separates vegetation from everything else but puts the burn scar (0.25) close to dry grass (0.35). NBR separates the burn scar, the only class with a clearly negative value, because the scar is dark in the near infrared and bright in SWIR2. NDWI and MNDWI are positive only for water. NDBI does not single out built-up land: bare soil scores about as high and the burn scar higher, because they, like roofs, are brighter in SWIR1 than in the near infrared, and dry grass is not far below. That is a property of the spectra chosen here, but it shows why an index is a one-dimensional projection of a spectrum and confuses any two surfaces that project to the same value (our reading of the table).

An index means something only at the processing level it was defined for. On raw counts, the indices reflect the instrument’s gains as much as the surface: EVI on counts reaches 10.3 for vegetation, far outside its usual range, because its constants assume reflectance. Radiance is no better for the shortwave indices. The solar irradiance in B12 is about a twelfth of that in B8, so SWIR2 radiance is small even over bright ground, and NBR on radiance is between 0.74 and 0.97 for every class, including the burn scar. NDVI on radiance is lower than on reflectance by a class-dependent amount, because the red band receives about 1.45 times the irradiance of the near-infrared band. This matters for older work: the 1974 vegetation index that Remote Sensing describes was computed from radiances. TOA reflectance recovered from the 12-bit counts matches the exact TOA reflectance to about 0.003, so quantization is not the issue. The atmosphere is. Between TOA and surface reflectance, NDVI changes by 0.11 for vegetation and by 0.12 for the burn scar, because path reflectance adds more to the dark red band than to the bright near-infrared band. EVI rises at TOA for vegetation, from 0.64 to 0.80, because this illustrative atmosphere brightens the blue band, which EVI subtracts with a large coefficient. A model trained on surface reflectance and applied to TOA reflectance, or the other way round, therefore sees shifted inputs, by amounts that depend on the class and the atmosphere (our reasoning from the table).

Misregistration creates errors only at edges, but they are large there. Shifting the near-infrared band by a tenth of a pixel already changes NDVI by up to 0.15 in some pixels; at 0.3 pixels the largest change is 0.41, and at one pixel almost 1. Away from class boundaries almost nothing changes: the mean interior error rounds to zero up to 0.6 pixels, and every pixel whose vegetation or water label flipped lies at an edge. The mean error over edge pixels grows almost in proportion to the shift, about 0.06 per pixel of shift, as expected when each shifted pixel takes a fraction of its neighbour’s near infrared equal to the shift (our reasoning from the box sampling). The largest changes occur where the near-infrared contrast is greatest and the red band is darkest, as where the river meets vegetation: a water pixel given a fraction of a field’s near infrared changes sign. The 0.3 and 0.6 pixel rows correspond to ESA’s registration tolerance between two 10 m bands and between a 10 m and a 20 m band. At those offsets, 0.3% and 0.6% of all pixels change their vegetation label in this scene, which has about a quarter of its pixels at edges.

The area estimate is biased, not just noisy. Flips in both directions might be expected to cancel, but the vegetation area grows by 0.76 percentage points at one pixel. The flips are asymmetric. A grass pixel that receives the near infrared of a neighbouring field gets an NDVI of $(0.40 - 0.12)/(0.40 + 0.12) = 0.54$ and crosses the 0.4 threshold, while a field pixel that receives the grass’s near infrared keeps $(0.25 - 0.04)/(0.25 + 0.04) = 0.72$ and stays vegetation (our calculation from the spectra). Whether misregistration inflates or deflates an area depends on the spectra and the threshold, so a bias of this kind should be expected rather than assumed away. The script was also run with seeds 1 to 4. Across seeds 0 to 4, at one pixel of shift, the mean NDVI change over edge pixels ranged from 0.053 to 0.056, the vegetation area grew by 0.58 to 0.78 percentage points, and between 92.4% and 100% of the flipped pixels lay at edges; at 0.3 pixels or less, every flip lay at an edge in all five scenes. Real images add noise, blur from the MTF and, for moving objects, the time delays between bands, all of which are absent here.

Hyperspectral Imaging in Outline

An imaging spectrometer records many narrow, contiguous bands, so that each pixel carries a near-continuous spectrum. NASA’s airborne AVIRIS, for example, delivers calibrated images of upwelling spectral radiance in 224 contiguous channels from 400 to 2500 nm; it has flown on NASA’s ER-2 at about 20 km altitude and on lower aircraft (NASA JPL, AVIRIS). In orbit, the German EnMAP mission, whose pages are published by DLR’s Earth Observation Center and which was launched in April 2022 on a Falcon 9, carries a pushbroom hyperspectral imager made of two prism spectrometers, one for the VNIR and one for the SWIR, sharing a three-mirror telescope. It covers 420–2450 nm with a spectral sampling distance of 6.5 nm in the VNIR and 10 nm in the SWIR, at 30 m over a 30 km swath, with at least 14 bits and a signal-to-noise ratio above 500 at 495 nm and above 150 at 2200 nm (EnMAP, Mission; Space Segment; Specifications).

The design changes what can be measured and what can go wrong. With contiguous narrow bands, absorption features of minerals, pigments and water can be resolved rather than averaged within a broad band, which is why EnMAP’s objectives emphasize geochemical, biochemical and biophysical parameters (EnMAP, Mission). Narrow bands collect little light, so swaths tend to be narrow and pixels coarse compared with multispectral instruments of similar size (our reasoning): EnMAP’s swath is a tenth of Sentinel-2’s. The registration requirements take a spectrometer-specific form. EnMAP specifies smile and keystone below 0.2 pixel; these are the spectral and spatial distortions of a dispersive spectrometer, in which a band’s wavelength or a pixel’s ground position drifts across the detector array, the hyperspectral counterparts of band-to-band misregistration (our gloss; EnMAP’s page gives only the figure). Hundreds of correlated bands also invite dimensionality reduction, for example with principal component analysis, and spectral unmixing, whose linear model Remote Sensing introduces.

Missions

The table summarizes the instruments of the main public missions from their operators’ pages, as of November 2026. Remote Sensing describes the programmes, their orbits and data policies.

Mission and instrument	Operator	Design	Bands	Resolution and swath	Quantization
Landsat 7 ETM+ (science mission April 1999 – April 2022)	NASA and USGS	whiskbroom, scan mirror with Scan Line Corrector	6 reflective, 1 thermal, 1 panchromatic	30 m; thermal 60 m; panchromatic 15 m; 185 km	8-bit
Landsat 8 OLI and TIRS (launched 11 Feb 2013)	NASA and USGS	pushbroom; TIRS with QWIP detectors	OLI: 8 multispectral and 1 panchromatic; TIRS: 2 thermal	30 m; panchromatic 15 m; thermal 100 m resampled to 30 m; 185 km	12-bit
Landsat 9 OLI-2 and TIRS-2 (launched 27 Sep 2021)	NASA and USGS	as Landsat 8	as Landsat 8	as Landsat 8	14-bit
Sentinel-2 MSI (2A 2015, 2B 2017, 2C 2024)	ESA, for the EU’s Copernicus	pushbroom, 12 staggered detector modules per focal plane	13, from 443 to 2190 nm	10, 20 and 60 m; 290 km	12-bit
EnMAP HSI (launched April 2022)	German mission; pages by DLR’s Earth Observation Center	pushbroom, two prism spectrometers	contiguous, 420–2450 nm	30 m; 30 km	≥ 14-bit

Landsat. NASA presents OLI, built by Ball Aerospace for both Landsat 8 and 9, as imaging nine spectral bands over a 185 km swath from 705 km. Its band 1 (0.433–0.453 µm) was added for coastal ocean colour and its band 9 for cirrus, and the refined near-infrared and cirrus bands closely match bands of the MODIS instruments (NASA, OLI); NASA’s band page adds that band 1 also serves to detect aerosols (NASA, Spectral Bands and Applications). NASA counts OLI and TIRS together as 11 bands, of which eight continue the “heritage bands” of earlier Landsats, and credits the seven-band Thematic Mapper of Landsats 4 and 5 with a band placement that guided all later Landsat sensors (NASA, Spectral Bands and Applications). TIRS images two thermal bands, 10.6–11.2 µm and 11.5–12.5 µm, which together let the surface temperature be separated from atmospheric effects. It is a pushbroom whose three arrays of 640 quantum well infrared photodetectors overlap to form an effective line of 1,850 pixels across the swath, cooled to 43 K by a cryocooler. Landsat 8’s TIRS suffered from stray light, corrected by an algorithm from February 2017, and Landsat 9’s TIRS-2 fixed it in the optics (NASA, TIRS). Landsat 9’s OLI-2 is a copy of OLI that quantizes to 14 bits instead of 12 (NASA, Landsat 9).

Sentinel-2. Sentinel-2C replaced Sentinel-2A in operation on 21 January 2025, alongside Sentinel-2B, and Sentinel-2D will succeed Sentinel-2B. Since March 2025, Sentinel-2A has flown an extension campaign 36° from Sentinel-2B, acquiring neighbouring orbits 10 minutes apart. The satellites’ design life is 7.25 years, with consumables for 12 years. All MSI data are systematically processed to Level-2A, globally since 13 December 2018 (ESA, S2 Mission).

Strengths

• Physical units. Calibrated radiance and reflectance make images comparable across dates, places and, with harmonization, sensors; NASA stresses that without consistent calibration images cannot be reliably compared through time (NASA, Landsat Calibration & Validation).
• Spectral information beyond the visible. Near-infrared, red-edge and shortwave infrared bands separate vegetation, water, soil, snow, cloud and burned ground in ways that colour photographs cannot.
• Long, consistent archives. Band continuity across Landsat instruments, and Sentinel-2’s design for continuity with Landsat and SPOT, support time series over decades.
• Wide swaths and regular revisits from pushbroom instruments.

Limitations

• Clouds and daylight. Passive optical sensors need sunlight and a clear view, unlike synthetic aperture radar.
• Atmosphere. TOA values include it, and surface reflectance rests on an estimate of it.
• Mixed resolutions and band timing. Bands must be resampled to be combined, are acquired at slightly different times and are registered only to fractions of the coarser pixel.
• Sensor-specific bands. Indices and models depend on the response functions and processing level they were built for.
• Spatial resolution. At 10 to 30 m, most objects are smaller than a pixel (Remote Sensing’s mixed pixels).

Typical Applications

Multispectral imagery is the main input for land cover classification and for change detection, and for the tasks that Remote Sensing surveys, from scene classification to cloud masking. Spectral indices serve specific measurements: water mapping with NDWI and MNDWI, burn severity with dNBR, which Key and Benson developed for fire effects over large areas, vegetation condition with NDVI and EVI, and crop monitoring in agriculture with indices and red-edge bands. NASA lists thermal applications of TIRS that include evapotranspiration and water use, urban heat, volcanic hazards, burned area and improved cloud detection in OLI data (NASA, TIRS). Pansharpening, which fuses the panchromatic and multispectral bands, is a form of super-resolution and is described in Remote Sensing.

Comparison with Other Sensors

• Synthetic aperture radar is active and works through cloud and at night, but records microwave backscatter, with speckle and side-looking geometry, rather than a spectrum of reflected sunlight.
• Hyperspectral imagers trade spatial coverage and signal for contiguous narrow bands, as described above.
• Colour cameras record three broad visible bands through a colour filter array, as the image sensor article describes. Multispectral scanners have separate bands extending into the infrared, and the tutorial of Natural Resources Canada notes that acquiring all bands through the same optical system alleviates the registration problems of multi-lens photographic systems.
• Laser scanning. LiDAR measures range with its own laser rather than reflected sunlight in several bands. It gives height and structure, which multispectral images lack, and is often combined with them.