Designing a Computer Vision System

A computer vision system is more than a model. It is a camera looking at a scene under some light, through a lens, at a certain distance; a computer that receives the frames within a time limit; software that turns pixels into a decision; data that the software was built and tested on; and an organization that keeps all of it working as the world changes. Designing one means working backwards from what the system must decide, how well and how fast, to each of those parts, because a weakness in any of them limits the whole. A detail the camera never resolved cannot be recovered by a better network, and a network that scores well on its test set can still fail on the first day in a new building. This article is the hub for the vision systems category: it follows the design from requirements to maintenance, summarizes each stage and links to the articles that treat it in depth. Getting a trained model into service is covered in deploying vision models, and the timing of a live camera stream in real-time vision pipelines.

Definition

Vision system design is the process of choosing and integrating the sensing, illumination, computing and software components of a system that extracts information from images, so that it meets stated requirements on what it decides, how accurately, how quickly, in what environment and at what cost of errors. Its stages are requirements, image acquisition (sensor, optics, lighting), processing (compute and model), data, validation and maintenance, and the process returns to earlier stages whenever a later one shows that a requirement is not met.

Intuition

Think of the system as a chain of conversions, each of which can only lose information: light from the scene becomes an optical image, the sensor samples it into pixels with noise, the model turns pixels into a decision, and the decision triggers an action. The design question at each link is how much loss the task can afford. A gauging station that must find a 0.1 mm error cannot afford pixels larger than a fraction of that; a warehouse robot that must notice a person cannot afford a long delay between seeing and stopping; a camera trap classifier cannot afford to have learned the background of the ten places it was trained in. Most of the arithmetic happens at the front of the chain, which is also, as a judgement rather than a measured result, where problems are cheapest to fix: changing a lens or adding a light costs less than collecting a new dataset, and far less than discovering the problem in the field.

The stages are not done once in order. In a study of software teams at Microsoft, Amershi et al. (2019) described machine-learning development as a nine-stage workflow, from model requirements through data collection, cleaning and labelling, feature engineering, training and evaluation to deployment and monitoring, and stressed that it is highly non-linear: evaluation and monitoring can send a team back to any earlier stage, for example when a large distribution shift between training data and real-world data appears. Vision adds stages in front of theirs, because the camera, optics and lighting are designed too.

Formal Definition

A design $d$ fixes the camera, lens, lighting, compute platform, model and decision threshold. It is acceptable if it meets the hard constraints and, among acceptable designs, the best one minimizes the expected cost of its decisions:

$$d^\star = \arg\min_{d}\; \mathbb{E}\big[\,c(\hat{y}_d, y)\,\big] \quad \text{subject to} \quad L(d) \le L_{\max},\;\; T(d) \ge T_{\min},\;\; s(d) \le s_{\max},$$

where $c(\hat{y}, y)$ is the cost of deciding $\hat{y}$ when the truth is $y$, $L$ is the latency from exposure to decision, $T$ the throughput in frames or parts per second, and $s(d)$ any physical limit such as the pixel size on the object, the motion blur, the power draw or the space for the camera. The expectation is over the images the system will actually see in service, not the images in its test set; the difference between the two is the subject of validation.

For a two-class decision, Elkan (2001, Section 1.3) showed that, when an error costs more than the corresponding correct decision (his “reasonableness” conditions), the expected cost is minimized by deciding “positive” whenever the estimated probability $p$ of the positive class satisfies $p \ge p^\star$, with

$$p^\star = \frac{c_{10} - c_{00}}{c_{10} - c_{00} + c_{01} - c_{11}},$$

where $c_{ij}$ is the cost of predicting class $i$ when the true class is $j$. When correct decisions cost nothing, $p^\star = c_{\mathrm{FP}} / (c_{\mathrm{FP}} + c_{\mathrm{FN}})$: if missing a defect costs 99 times as much as rejecting a good part, parts should be rejected from $p = 0.01$ on, not from 0.5. For the Bayesian and decision-tree learners he studied, Elkan recommends training on the data as given and computing the cost-optimal decision from the classifier’s probability estimates, smoothed where necessary (Section 5); the rule is only as good as those probabilities, which must be calibrated (expected calibration error measures how far they are from it).

Requirements

Requirements turn a wish (“check the parts”, “find people near the forklift”) into numbers that each later stage can be designed against. The questions are the same for most systems:

• Task. What exactly is decided, and what is the output: a class per image (image classification), boxes (object detection), a label per pixel (semantic segmentation), a measurement in millimetres, a pose? The smallest thing that must be seen, and the largest scene it must be seen in, set the resolution.
• Accuracy and its metric. The metric must match the decision: precision and recall at a fixed operating point for a detector that triggers an action, the escape rate (defective parts accepted) and false reject rate (good parts rejected) for inspection, an error in millimetres with a stated repeatability for gauging. The confusion matrix is the common ground for all of them.
• Cost of errors. False positives and false negatives rarely cost the same. The ratio fixes the operating threshold, as above, and it can decide whether a fully automatic system is acceptable at all or whether uncertain cases go to a person.
• Latency and throughput. How long after the event the decision is still useful, and how many frames or parts per second must be handled. They are different requirements: a pipeline can process every frame and still answer too late. The real-time vision pipelines article defines frame budgets, keeps the two quantities apart and shows how frames waiting in queues add latency.
• Environment. The range of lighting (daylight, night, flicker from mains-powered lamps), weather, temperature, vibration, dust, the variety of objects and backgrounds, and how much of this the design can control. A factory cell can be enclosed and lit; a street cannot.
• Constraints. Power, size, network connectivity, privacy (whether images may leave the device; see edge inference), the people available to label and maintain, and regulation in the domain.

Requirements are worth writing as testable statements (“escape rate below 0.1% at 95% confidence on parts from all three production lines”), because the acceptance test at the end checks exactly those statements.

Sensor, Optics and Lighting

Image acquisition is designed, not taken as given. The digital camera article describes the camera as a system, the image sensor article the chip, and image formation the physics from light to pixel value; the choices a design makes are these:

• Resolution and field of view. The smallest feature sets the size of a pixel on the object, the scene sets the field of view, and their ratio sets the number of pixels, as the example below works out. Resizing the image before the model can throw away exactly the detail that was paid for.
• Exposure and motion. The exposure time sets both the light collected and the motion blur; shortening it to freeze motion costs light, which must come back from a wider aperture (less depth of field), more gain (more noise) or more illumination (exposure treats the trade-off). A rolling shutter exposes rows at different times and skews moving objects, as the image sensor article derives, which favours a global shutter for fast motion.
• Sensor type. Area-scan or line-scan, monochrome or colour (a colour filter array samples each colour at only part of the pixels), visible or infrared, or a depth sensor such as LiDAR or a stereo pair when geometry matters more than appearance. Cameras whose datasheets follow EMVA 1288 report their sensitivity, temporal noise, linearity, nonuniformity, defect pixels and dark current measured by one documented method (EMVA 1288, Table 1); the standard names as its main beneficiaries those who wish to compare cameras or to calculate system performance from the specifications of a sensor or camera.
• Optics. The focal length and working distance set the field of view; the aperture sets light, depth of field and diffraction; real lenses add distortion. For measurement, an ordinary lens changes magnification with distance, which a telecentric lens avoids, at the price of a front element at least as large as the field of view (industrial visual inspection describes both). Camera calibration then maps pixels to millimetres and removes distortion.
• Lighting. Where the scene can be lit, the lighting decides which features have contrast. Back light gives silhouettes for measuring outlines, low-angle dark-field light makes scratches bright, and diffuse dome light suppresses reflections on shiny curved parts; the industrial visual inspection article describes these geometries from lighting makers’ documentation. On a production line the light is often strobed in time with the exposure, and the camera is triggered by the part’s arrival and wired to the machines that act on its result; industrial machine vision covers these system decisions. Where the scene cannot be lit, the design must cope with whatever light there is, and the range of that light becomes a requirement for the data and the tests.

Compute and Model

The model and the hardware it runs on are chosen together, against the latency, throughput, power and connectivity requirements:

• Where to compute. On the device, on a computer next to the camera, or on a server reached over a network. Edge inference covers the constraints of compute, memory, power and heat on the device and how latency and energy are measured; deploying vision models covers exporting, packaging and serving a model, and keeping its preprocessing identical to training.
• Classical or learned. As a rule of thumb (ours, not a cited finding): when the images are designed, with fixed lighting and a fixed pose, classical methods such as thresholding, edge detection, mathematical morphology and template matching are fast, explainable and need few examples; libraries such as OpenCV provide them. When appearance varies (natural scenes, deformable or varied objects), the same rule points to learned models, adapted from a pretrained network by transfer learning and fine-tuning when labelled images are scarce. The two combine: a classical stage can locate the part and a learned stage judge it.
• Fitting the budget. A model too slow for its frame budget can be made smaller or cheaper by quantization and the other methods of model compression, exported through a format such as ONNX to a runtime built for the target hardware, and run in a pipeline that overlaps capture, inference and output. Frameworks such as PyTorch and TensorFlow are where models are trained; the runtime in service is often a different program.

Data

A learned model is specified by its data as much as by its code. The data must cover the conditions in the requirements (every product variant, line, camera, season, lighting state), and its labels must follow written guidelines that match the decision the system makes. Image annotation covers label types, guidelines, agreement between annotators, quality control and cost; data augmentation extends coverage synthetically but cannot add conditions it does not model; dataset bias describes how the way images are chosen makes a dataset unrepresentative of the world.

The split between training and test data must mirror the deployment. If the system will run at new sites, the test set must contain sites absent from training, and frames of one video must not be divided between training and test sets, which inflates the score (training, validation and test sets). Amershi et al. (2019) identified discovering, managing and versioning data as much more complex than the corresponding work in other kinds of software engineering, and Sculley et al. (2015) argued that data dependencies cost more than code dependencies and are harder to detect, because no tooling comparable to the static analysis that compilers and linkers do for code tracks them.

Validation in the Field

A test set drawn from the same sources as the training data measures in-distribution performance. Deployment brings distribution shift: new places, cameras, lighting, products or populations. The evidence that the gap is large is consistent:

• Beery, Van Horn and Perona (2018) trained classifiers on camera-trap images from 10 locations and tested them both at those locations and at 9 others. With empty images removed and no sequence information, top-1 error on whole images rose from 19.06% at the training locations to 41.04% at new ones, and on cropped animal boxes from 8.14% to 19.56% (their Table 1). Camera traps are fixed, so the background changes little across a location’s images; a model can then lean on backgrounds that do not travel to new sites (our reading).
• The WILDS benchmark (Koh et al., 2021) collected ten datasets with shifts that arise in practice, across hospitals, camera traps, time and regions, and found standard training substantially worse out of distribution than in distribution on every one. On its camera-trap dataset, where the test images come from traps absent from the more than 200 used for training, macro F1 fell from 47.0 on test images from the training traps to 31.0 on the new ones (their Table 1).
• Even a careful replication of a dataset’s collection procedure can lower accuracy, as the ImageNetV2 paper found for every model it tested.

Validation therefore has three parts. First, an offline test on held-out conditions: sites, cameras, dates or lines kept out of training, which estimates how performance drops under the shifts that are expected. Second, stress tests for the environment requirements: Hendrycks and Dietterich (2019) built ImageNet-C from 15 algorithmic corruptions in four groups (noise, blur, weather and digital), each at five severities, and the same idea of rendering a requirement’s conditions, such as blur at the maximum allowed speed or noise at the lowest light, lets a team check each one deliberately (our suggestion). Third, an acceptance test in the field: the system runs on the real line or site, for example first in shadow, with its decisions recorded but not acted on, and its errors are counted against the written requirements. Statistical significance in evaluation gives the confidence intervals for such counts; the example below shows why rare failures need large tests.

Maintenance and Monitoring

Deployment is the start of the system’s life, not the end of the project. Sculley et al. (2015) observed that ML systems are fast to build and expensive to maintain, and named risks that matter for vision: models entangle their inputs, so that changing anything changes everything (their CACE principle); manually set decision thresholds may become invalid when a model is updated on new data, and they point to learning thresholds on held-out validation data as one mitigation; and because the external world is rarely stable, unit and end-to-end tests are not enough without live monitoring. As starting points for monitoring they propose comparing the distribution of predicted labels with that of observed labels, setting limits on how often the system may take an action, and monitoring the upstream producers of its data. In a vision system the upstream producers include the camera and its firmware, the lights and the lens, which can age, drift out of focus or get dirty without any software change (our extension of their point).

The deploying vision models article covers the software side: input and prediction statistics, latency, labelled samples of logged images, canary releases and rollback. The physical side needs its own checks; one option (our suggestion) is a reference target imaged at fixed intervals to detect changes in focus, brightness or calibration. When monitoring finds a shift, the loop of Amershi et al. (2019) closes: new data is collected and labelled, the model is retrained and re-validated, and the thresholds are set again.

Example: Requirement Arithmetic for a Gauging Station

A station must check the positions of holes in a 40 × 30 mm bracket to ±0.1 mm. Parts ride a conveyor at 200 mm/s, spaced 100 mm apart, and arrive within ±5 mm of their nominal position. Two design rules enter as stated assumptions, not as cited standards: the tolerance band (0.2 mm) should span at least 10 pixels, so that edge-localization error is a small part of it, and motion blur should stay under 1 pixel. The script derives the field of view, the pixel size and sensor resolution, the focal length from the thin-lens equation, the perspective error, the longest exposure, the trigger and data rates, and the size of an acceptance test.

The thin-lens equation $1/z_o + 1/z_i = 1/f$ relates the object distance $z_o$, image distance $z_i$ and focal length $f$, and the magnification is $|m| = z_i/z_o$ (Ling, Sanny & Moebs, 2016, Section 2.4, Eqs. 2.19 and 2.22). Eliminating $z_i$ gives the focal length that images a field of view onto a sensor at a working distance,

$$m = \frac{\text{sensor size}}{\text{field of view}}, \qquad f = \frac{m\, z_o}{1 + m}, \qquad z_o = \frac{f\,(1+m)}{m},$$

a derivation, not a quoted formula. The pinhole shortcut $f = m\,z_o$ assumes the image plane sits at the focal length, which holds only for distant objects. Motion blur in pixels is the distance the part moves during the exposure divided by the pixel size on the part, so the longest exposure is $t_{\max} = b_{\max}\,p/v$ for a blur budget $b_{\max}$ in pixels, pixel size $p$ and speed $v$ (it follows from the definition of blur; exposure gives the same relation in image units). For the acceptance test, if a system misses a defective part with probability $p_e$, the chance of seeing no misses in $n$ defective parts is $(1 - p_e)^n$; requiring it to be at most $\alpha$ gives $n \ge \ln\alpha / \ln(1 - p_e)$, about $3/p_e$ for $\alpha = 0.05$, the binomial “rule of three” that industrial visual inspection states for test sets.

import numpy as np

# --- Requirements (the inputs a customer or process engineer gives) ---
part_w, part_h = 40.0, 30.0   # part size, mm
placement = 5.0               # part position varies by +/- 5 mm on the conveyor
tol = 0.1                     # hole position tolerance, +/- mm
px_per_band = 10              # assumption: the tolerance band spans >= 10 pixels
speed = 200.0                 # conveyor speed, mm/s
pitch_parts = 100.0           # distance between parts on the belt, mm
blur_max_px = 1.0             # assumption: at most 1 pixel of motion blur
wd_target = 300.0             # desired lens-to-part distance, mm

# Field of view and the pixel size the tolerance demands
fov_w, fov_h = part_w + 2 * placement, part_h + 2 * placement
px_obj_max = 2 * tol / px_per_band             # mm on the part per pixel
need_w, need_h = np.ceil(fov_w / px_obj_max), np.ceil(fov_h / px_obj_max)
print(f"field of view {fov_w:.0f} x {fov_h:.0f} mm, pixel on part <= "
      f"{px_obj_max:.3f} mm -> need {need_w:.0f} x {need_h:.0f} pixels")

# Pick the first candidate sensor (width px, height px, pixel pitch um) that fits
candidates = [(1920, 1200, 5.86), (2448, 2048, 3.45), (4096, 3000, 3.45)]
for nw, nh, pitch_um in candidates:
    ok = nw >= need_w and nh >= need_h
    print(f"  sensor {nw} x {nh} px, {pitch_um} um: {'fits' if ok else 'too few pixels'}")
    if ok:
        break
sw, sh = nw * pitch_um / 1000, nh * pitch_um / 1000   # sensor size, mm

# Magnification: the FOV must fit in both directions, so the tighter one wins
m = min(sw / fov_w, sh / fov_h)
px_obj = pitch_um / 1000 / m
print(f"sensor {sw:.2f} x {sh:.2f} mm, magnification {m:.4f}, actual FOV "
      f"{sw / m:.1f} x {sh / m:.1f} mm, {px_obj * 1000:.2f} um per pixel, "
      f"{2 * tol / px_obj:.1f} pixels across the tolerance band")

# Focal length from the thin-lens law 1/f = 1/z_o + 1/z_i with m = z_i / z_o
f_thin = wd_target * m / (1 + m)
f_pin = wd_target * m                         # pinhole guess with image plane at f
print(f"focal length for {wd_target:.0f} mm: thin lens {f_thin:.1f} mm "
      f"(pinhole guess {f_pin:.1f} mm)")
f_lens = 50.0                                 # nearest focal length on the shelf
wd = f_lens * (1 + m) / m
print(f"with a {f_lens:.0f} mm lens the part must sit at {wd:.0f} mm")

# Perspective: a feature x mm off-axis moves when the part sits dh higher
x_off, dh = 20.0, 1.0
shift = x_off * dh / (wd - dh)
print(f"1 mm height change shifts a point {x_off:.0f} mm off-axis by "
      f"{shift:.3f} mm = {shift / px_obj:.1f} px (tolerance +/-{tol} mm)")

# Exposure time from speed and allowed blur; trigger rate and data rate
t_max = blur_max_px * px_obj / speed
rate = speed / pitch_parts
print(f"max exposure {t_max * 1e6:.1f} us for {blur_max_px:.0f} px blur; "
      f"{rate:.0f} parts/s; {nw * nh * rate / 1e6:.1f} MB/s at 8 bits per pixel")

# Acceptance test: zero escapes in n defective parts bounds the escape rate
for p_esc in (0.01, 0.001):
    n = np.ceil(np.log(0.05) / np.log(1 - p_esc))
    print(f"show escape rate < {p_esc:.1%} at 95% confidence: "
          f"{n:.0f} defective parts, none missed")

Output (NumPy 2.3.5; the script is deterministic and two runs printed identical output):

field of view 50 x 40 mm, pixel on part <= 0.020 mm -> need 2500 x 2000 pixels
  sensor 1920 x 1200 px, 5.86 um: too few pixels
  sensor 2448 x 2048 px, 3.45 um: too few pixels
  sensor 4096 x 3000 px, 3.45 um: fits
sensor 14.13 x 10.35 mm, magnification 0.2587, actual FOV 54.6 x 40.0 mm, 13.33 um per pixel, 15.0 pixels across the tolerance band
focal length for 300 mm: thin lens 61.7 mm (pinhole guess 77.6 mm)
with a 50 mm lens the part must sit at 243 mm
1 mm height change shifts a point 20 mm off-axis by 0.083 mm = 6.2 px (tolerance +/-0.1 mm)
max exposure 66.7 us for 1 px blur; 2 parts/s; 24.6 MB/s at 8 bits per pixel
show escape rate < 1.0% at 95% confidence: 299 defective parts, none missed
show escape rate < 0.1% at 95% confidence: 2995 defective parts, none missed

The sensor sizes are illustrative formats, not products. The middle one misses the width requirement by 52 pixels, so the design steps up to the next format; its wider shape (4096:3000, about 1.37:1) differs from the 5:4 field of view, so the height decides the magnification and the width gains 4.6 mm of unused view, while the tolerance band gets 15 pixels instead of the minimum 10. At this close range the pinhole shortcut overestimates the focal length by about a quarter, and the lens that is actually available fixes the working distance rather than the other way round.

The perspective line is the finding that changes the design. With an ordinary lens at 243 mm, a part sitting 1 mm higher on the conveyor moves a hole near its edge by 0.083 mm, most of the ±0.1 mm tolerance, so either the part height must be held far tighter than 1 mm, or the station needs a telecentric lens, whose magnification does not depend on distance. The 66.7 µs exposure limit is short enough that the light, not the camera, becomes the next problem: collecting enough photons in that time needs a bright or strobed source, which is why lighting is designed alongside the optics. At 2 parts per second the frame budget is 500 ms, which leaves room for a learned model; the 24.6 MB/s stream is what the camera interface and the computer must carry. The last two lines explain why acceptance tests of inspection systems are hard: showing an escape rate below 0.1% takes about 3,000 defective parts with none missed, and defective parts are by design rare, so a test of this size means collecting defects over a long period or producing them on purpose (our reasoning).

The blur relation can be checked on a rendered image. The script below renders one row of a dark-to-bright edge moving across the sensor during the exposure, with each pixel integrating the exact area the bright side covers at 2,000 instants, and estimates the blur length from the spread of the edge’s gradient: a uniform smear of length $b$ has variance $b^2/12$, and the pixel’s own width and the finite difference add $2/12$ (our derivation).

import numpy as np

def moving_edge(blur_px, n_px=64, steps=2000):
    """Row of a dark-to-bright step edge that moves blur_px pixels during
    the exposure. Each pixel integrates the light that falls on its area
    (exact box coverage) at each of `steps` instants of the exposure."""
    centers = np.arange(n_px) + 0.5
    edge = 20.0 + blur_px * (np.arange(steps) + 0.5) / steps   # edge position
    cover = np.clip(centers[None, :] + 0.5 - edge[:, None], 0.0, 1.0)
    return cover.mean(axis=0)                                  # 0 = dark, 1 = bright

def blur_width(row):
    """Motion-blur length from the spread of the edge's gradient: a box of
    length b has variance b^2/12, and pixel area plus finite difference add 2/12."""
    d = np.diff(row)
    d = d / d.sum()
    x = np.arange(d.size)
    var = (d * x**2).sum() - (d * x).sum() ** 2
    return np.sqrt(max(12 * var - 2, 0.0))

px_obj = 40.0 / 3000             # mm on the part per pixel (40 mm over 3000 rows)
speed = 200.0                    # mm/s
t_max = px_obj / speed           # exposure for 1 pixel of blur
for k in (1, 2, 4, 8):
    t = k * t_max
    row = moving_edge(speed * t / px_obj)
    mid = np.interp(0.5, row, np.arange(row.size) + 0.5)   # 50% crossing
    print(f"exposure {t * 1e6:6.1f} us: v*t = {speed * t / px_obj:4.2f} px, "
          f"measured blur {blur_width(row):4.2f} px, "
          f"edge found {mid - 20.0:4.2f} px past its start")

Output (NumPy 2.3.5, identical over two runs):

exposure   66.7 us: v*t = 1.00 px, measured blur 1.00 px, edge found 0.50 px past its start
exposure  133.3 us: v*t = 2.00 px, measured blur 2.00 px, edge found 1.00 px past its start
exposure  266.7 us: v*t = 4.00 px, measured blur 4.00 px, edge found 2.00 px past its start
exposure  533.3 us: v*t = 8.00 px, measured blur 8.00 px, edge found 4.00 px past its start

The blur measured on the rendered edge matches $v\,t$ divided by the pixel size at every exposure, so the budget computed on paper holds for an ideal camera; a real lens adds its own blur, which the same measurement on a real image would include. The second column of numbers carries a separate lesson for measurement: the edge is found at its position half-way through the exposure, half the blur length past where it was when the exposure began. A system that pairs the image with an encoder reading or timestamp taken at the start of exposure makes a systematic error of half the blur, which is negligible at 1 pixel of blur but not at 8; the real-time vision pipelines article discusses which instant a timestamp marks.

Common Misconceptions

• “A better model will make up for poor images.” A model can only use the information the images contain. Detail smaller than a pixel, contrast the lighting never created and edges smeared by motion are lost before any software runs; the cheapest fix is usually optical (our reasoning).
• “Test-set accuracy is the accuracy in service.” It is the accuracy on images like the test set. The camera-trap and WILDS results above show how much a change of site alone can cost, which is why the test split must hold out the conditions the system will meet.
• “More pixels are always better.” Each extra pixel adds data to move and compute, and at a fixed sensor size and irradiance smaller pixels collect fewer photons, since the mean photon count per pixel is proportional to its area (EMVA 1288, Eq. 3). The example picks the smallest format that meets the tolerance.
• “The focal length is magnification times distance.” That pinhole relation holds for distant scenes; at close range the thin-lens equation gives a noticeably shorter focal length, about 20% shorter in the example.
• “Once validated, the system is done.” Sculley et al. (2015) describe ML systems as cheap to build and expensive to maintain, and the camera hardware drifts as well. Monitoring, scheduled re-validation and a way to retrain belong to the design.

Where It Is Used

The process applies to any vision system that is deployed, whether or not its steps are written down. Factory inspection and gauging, where the images can be designed and the requirements are strict, show the process most explicitly; reading text on labels and documents (optical character recognition), face recognition at access points, augmented reality on phones and headsets, and robots and vehicles that localize with SLAM, steer by camera feedback (visual servoing) or combine cameras with LiDAR and radar (sensor fusion for autonomous driving) face the same chain from requirements to monitoring with less control over their scenes. Camera-trap monitoring in ecology, the subject of the shift studies above, is an example where the environment is fixed per camera but different at every new site.