Human Visual System

The human visual system is the chain of organs that turns light into seeing: the optics of the eye, which form an image on the retina; the photoreceptors, which sample that image; the retinal neurons, which transform the samples into a set of parallel signals carried by the optic nerve; the lateral geniculate nucleus (LGN) in the thalamus, which relays them; and the visual cortex, starting with the primary visual cortex (V1), where the signals reach many specialised areas. Computer vision has borrowed from this system repeatedly. Torralba, Isola and Freeman (2024, §1.3.5) list red-green-blue image encoding, filter-based image representations, neural networks and attention modulation as approaches inspired by neuroscience, and the computer vision article traces Marr’s levels of analysis and the line from Hubel and Wiesel’s cells to convolutional networks. This article describes the stages themselves, with the numbers vision science gives for them, and computes the transfer function of the classic model of a retinal receptive field.

Most numbers below come from Brian Wandell’s Foundations of Vision in its online edition, which is revised past the 1995 book (it cites work from as late as 2022), and from its appendix of useful numbers. Much of what is known about the visual pathway beyond the eye was measured in cats and monkeys; Wandell (ch. 6) stresses that species differ, and notes that removing V1 does not blind a cat, while its loss is devastating to all visual function in humans.

Definition

The human visual system is the eye together with the neural pathways and brain areas that process the signals of its photoreceptors: the retina, the optic nerve and tract, the LGN, and the visual areas of the cortex. In vision science it is studied at several levels: the physics of image formation and sampling, the electrical responses of single neurons (each described by its receptive field, the region of the visual field where light changes the neuron’s response), and the behaviour of whole observers, measured in psychophysical experiments such as contrast-detection thresholds.

Intuition

A digital camera forms an image with a lens, samples it with a sensor and stores the samples. The eye forms an image too, but nothing downstream stores it. The retina samples densely only in a small central region, the fovea; it does not send the photoreceptor values on, but transforms them within a few synapses into signals that report local contrast, the change in light relative to the local average, and it splits them into parallel streams, each sent to a different destination. Wandell (ch. 5) argues that coding contrast rather than absolute light is what lets neurons with a small response range work while the ambient light varies over many orders of magnitude. Further along, cortical neurons respond selectively to bars at one orientation, to motion in one direction or to stimuli in one eye. The visual system is better thought of as a set of measurements made on the retinal image, each tuned for some purpose, than as a picture passed to the brain: the opening page of Wandell’s book calls perception “an interpretation of the retinal image, not a description”.

The Eye as an Optical Instrument

The eye’s optics are a two-element system: the cornea supplies about 43 dioptres and the relaxed lens about 20, for about 60 dioptres for the whole eye (Wandell, Useful numbers). Accommodation, the lens changing shape to focus near objects, changes the power by about 8 dioptres, and the pupil ranges from 2 to 8 mm in diameter. Distances on the retina are usually given as visual angle: one degree corresponds to about 0.3 mm on the retina, and Wandell (ch. 3) computes the angle subtended by a cone from a distance of 17 mm between the effective centre of the optics and the retina. Their general optics are those of any camera (image formation, point spread function).

The optics are not perfect, and they blur the image before it is sampled. The eye’s axial chromatic aberration across the visible spectrum is about 2 dioptres (Useful numbers), so the short-wavelength part of the image is blurred more than the rest; Wandell (ch. 3) notes that there is very little contrast beyond 2–3 cycles per degree in the short-wavelength part of the spectrum, and that a 60 cycles-per-degree sinusoid cannot exceed 10% contrast after passing through the optics. Spatial frequencies on the retina are measured in cycles per degree (cpd) of visual angle; every frequency in this article uses that unit.

The Photoreceptor Mosaic

Each retina has about $10^8$ rods and $5 \times 10^6$ cones (Useful numbers). Rods serve vision at low (scotopic) light levels and cones at higher (photopic) levels; both work in the intermediate mesopic range (Wandell, ch. 3). Rods respond to single photons, but the signals of many rods converge onto single neurons, which raises sensitivity and makes rod vision poor at resolving detail. The fovea holds the highest concentration of cones and no rods at all, which is why, as Wandell explains, a dim star fixated directly can vanish: the all-cone fovea is not sensitive enough to see it, and it reappears when placed in the periphery, where rods detect it. The foveal cones do the opposite of the rods: several neurons encode the signal from each foveal cone. Over the whole retina, Useful numbers gives 125 receptors per ganglion cell, the neurons whose axons form the optic nerve, and about $1.5 \times 10^6$ optic nerve fibres per retina.

The cones come in three kinds, called L, M and S after the long, middle and short wavelengths of their peak sensitivity; how three spectral sensitivities reduce every light to three numbers is the subject of colorimetry and color spaces. The three kinds sample the image at different densities. In the central fovea the L and M cones are tightly packed in a triangular grid with a spacing of about 0.5 minutes of arc, about 120 cones per degree, while the S cones are about 10 minutes of arc apart and absent from the very centre (Wandell, ch. 3; Useful numbers). Wandell relates the sparse S mosaic to the strong chromatic blur of the short-wavelength image: there is little short-wavelength detail to sample.

The mosaic is a sampling array, and it obeys the rules of sampling and aliasing. For a uniform array with $N$ samples per degree, gratings at frequencies equally far above and below the Nyquist frequency $N/2$ give identical samples, and Wandell (ch. 3) emphasises that once two signals are aliased, no later processing can separate them. At about 120 cones per degree the foveal Nyquist frequency is about 60 cpd (our arithmetic). In ordinary viewing the optical blur keeps such frequencies off the retina, so, in Wandell’s words, optical defocus protects the fovea from aliasing. Visual interferometry bypasses the optics: two coherent beams interfere on the retina and produce a sinusoidal grating of high contrast at frequencies the lens could not pass. As Wandell describes it, observers viewing such gratings well above 60 cpd see coarser, wavy patterns that vary across the visual field, as Helmholtz and later Byram had already reported for fine patterns, and Williams’ group replicated and extended these observations. Wandell concludes from the aliasing measurements that the foveal grid is regular, at about 120 cones per degree, in agreement with anatomical images reported by Curcio and colleagues. Because cone density falls with distance from the fovea, aliasing begins at lower frequencies in the periphery.

The Retina

The retina is a sheet of neural tissue lining the back of the eye, with three layers of cell bodies and two layers of synaptic connections over most of its extent; in the fovea the neurons are pushed aside, out of the light path (Wandell, ch. 5). Ramón y Cajal identified its basic anatomical structure (Wandell, ch. 5); Torralba et al. (§1.3.5) date his isolation of individual neurons with Golgi’s stain to 1890, and Cajal and Golgi shared the 1906 Nobel Prize. Wandell (ch. 5) names five basic classes of retinal neuron. The photoreceptors connect to bipolar and horizontal cells in the outer plexiform layer; the bipolar cells connect to the ganglion cells in the inner plexiform layer, where amacrine cells also act; and since only bipolar cells link the two synaptic layers, every visual signal passes through them. The ganglion cell axons are the retina’s only output. They leave the eye at the optic disk, where there are no photoreceptors: the blind spot.

The retina does not send one copy of the image but many. Wandell (ch. 5) cites an estimate by Rodieck and colleagues of as many as twenty pathways leaving the retina, each sent to a different place in the brain, and about 15 to 20 kinds of bipolar cell contacting each cone. Wandell (ch. 6) calls two of them the best understood streams. The midget ganglion cells, with small dendritic fields, begin the parvocellular pathway, and the parasol cells, with larger fields, begin the magnocellular pathway; both end in their own layers of the LGN. The rods have a pathway of their own for a few synapses: in Wandell’s account a single rod bipolar cell may pool the signals of 1,500 rods before the rod signal merges with a cone-initiated pathway.

Receptive fields and centre–surround organization. Wandell (ch. 5) credits Hartline (1938) with first using the receptive-field concept in vision; Torralba et al. (§1.3.5) write that Hartline popularized a concept introduced by Sherrington in 1906 for touch. As Wandell and Torralba et al. describe it, Kuffler (1953), recording from cat ganglion cells while flashing small spots of light at different positions, found the concentric organization now called centre–surround. In an on-centre cell, light in a small central region increases the firing rate and light in the surrounding annulus decreases it; off-centre cells, about the other half of the ganglion cells in Wandell’s account, do the reverse. For certain cat ganglion cells, in Wandell’s words, linearity holds rather well: Enroth-Cugell and Pinto (1970), as Wandell describes them, found that the response to a central spot and a surrounding annulus shown together matched the sum of the two separate responses.

The difference-of-Gaussians model. In the mid-1960s Rodieck (1965) and Enroth-Cugell and Robson (1966) introduced the linear model now called, in Wandell’s term, the Difference of Gaussian model (Wandell, ch. 5). It sums two mechanisms, a centre and a surround, whose spatial sensitivities both follow Gaussian curves; the surround covers a larger region, including the centre, and has the opposite sign. In Wandell’s account the surround’s signal is also delayed in time, so that the cell as a whole is not space-time separable even though each mechanism is. Wandell notes that the model’s ideas spread to psychophysics and to edge detection and segmentation in computer vision, and he stresses that the two mechanisms are theoretical: the model describes the measurements without proving that separate centre and surround pathways exist.

The model predicts a cell’s response to a sinusoidal grating. Wandell (ch. 5) explains the shape of a centre–surround neuron’s contrast sensitivity function as follows. At low spatial frequencies a bright bar covers both centre and surround, which cancel. At an intermediate frequency the bright bars fall on the centre and the dark bars on the surround, and the response is strongest. At high frequencies the centre itself averages light and dark. The falloff at low frequencies measures the strength of the surround, the falloff at high frequencies the size of the centre, and the most effective frequency the overall size of the field. His example is a parvocellular LGN neuron of the monkey (data of Derrington and Lennie) that responds best near 5 cpd. The curve also changes with light level: in cat ganglion cells measured by Enroth-Cugell and Robson (1966), it is band-pass on bright backgrounds and becomes low-pass, with little sign of the surround, at low mean intensities. Wandell reads this as a form of adaptation: when photons are scarce the neuron sums all of them, and when they are plentiful it signals variation within its field. In another measurement he describes (Enroth-Cugell and colleagues, 1977), the threshold intensity of a test flash grew in proportion to the background intensity, with a log–log slope close to one.

From the Retina to the Cortex

The lateral geniculate nucleus. The ganglion cell axons from both eyes meet at the optic chiasm, where they are sorted so that each side of the brain receives the opposite half of the visual field from both eyes (Wandell, ch. 6). Wandell cites an estimate that 90% of monkey retinal ganglion cells project to the LGN, and about 10% to the superior colliculus in the midbrain (ch. 5). Torralba et al. (§1.3.5) describe the LGN as six layers of neurons with centre–surround receptive fields that keep the two eyes’ signals separate, so that its neurons are monocular. Its role is not fully understood (they mention temporal decorrelation, attention modulation and saccadic suppression as possibilities), and they report that only 5% of its input connections come from the retina and 95% are feedback from V1 and other areas. Neurons keep their retinal neighbourhoods: the LGN layers, and V1 after them, are retinotopic, laid out as a map of the visual field (Wandell, ch. 6).

Primary visual cortex. In the primate, most of the visual signal reaches one area of the occipital lobe, V1, also called striate cortex after a band of myelinated axons visible to the naked eye (the stria of Gennari). Wandell (ch. 6) gives it roughly $1.5 \times 10^8$ neurons, against about $10^6$ in the LGN. V1 has six layers. The magnocellular input arrives in layer 4Cα and the parvocellular input in layer 4Cβ; layer 6 sends a large output back to the LGN; and signals from the two eyes stay separated in layer 4C, in alternating bands called ocular dominance columns, before converging on binocular neurons in the superficial layers. Torralba et al. (§1.3.5) add a third LGN channel, the koniocellular pathway, which projects to the colour-sensitive “blobs” that Livingstone and Hubel described in the upper layers.

Torralba et al. (§1.3.5) tell how Hubel invented a microelectrode for recording single cortical cells between 1955 and 1958 and then worked with Wiesel in Kuffler’s laboratory. In their 1959 study, from 45 neurons in the striate cortex of an anesthetized cat, Hubel and Wiesel found cells driven best by a bar at a particular orientation in the centre of the receptive field; Torralba et al. describe them classifying cells as simple, complex or hypercomplex, and their cortical recordings led to the 1981 Nobel Prize. Neurons in V1 are organized in columns: along one direction across the cortex the preferred orientation changes smoothly (orientation columns), along the perpendicular direction the preferred eye alternates (ocular dominance columns), and a region of about 1 × 3 mm, which Torralba et al. call a hypercolumn, covers all orientations and both eyes for a small part of the visual field. Wandell (ch. 6) adds the distinction that matters most for models. Simple cells, with adjacent excitatory and inhibitory regions elongated along the preferred orientation, are approximately linear for moderate contrasts once their low spontaneous rate is taken into account, so their responses can be predicted from their contrast sensitivity. Complex cells are orientation selective too, but nonlinear: to a contrast-reversing grating they respond in both phases, at twice the stimulus frequency. Some neurons are also direction selective, responding to motion one way and not the other. Even simple cells show a nonlinearity, contrast normalization, at higher contrasts.

Beyond V1. More than twenty cortical areas receive strong visual input (Wandell, ch. 6). Torralba et al. (§1.3.5) describe two main streams, ventral and dorsal, and specialized areas for motion (area MT), faces (the fusiform face area) and object recognition (inferotemporal cortex, IT). In Wandell’s account the magnocellular stream reaches area MT through layer 4B, where many neurons are direction selective, which supports the hypothesis that MT is involved in motion perception; another branch of the magnocellular signal merges with the parvocellular one in the superficial layers of V1. Wandell is cautious about assigning one percept to each area. He calls Livingstone and Hubel’s anatomical–perceptual diagram, which maps cortical streams onto features such as colour, form, motion and depth, a speculative model, and he argues that the neuron doctrine behind such mappings (Barlow, 1972: that a neuron’s receptive field describes the percept its activity causes) gets little support from what is known about the periphery.

Contrast Sensitivity of the Observer

The same contrast sensitivity function is measured for whole observers, as the reciprocal of the threshold contrast at which a grating is just visible; this observer-level curve is not the single-neuron curve above, though the two are compared. Wandell (ch. 7) traces the approach to Schade (1956), whose model of the eye, built as a photoelectric analog to predict whether changes to a display would be noticed, may be the earliest computational model of human vision (in the words of Wandell’s figure caption). Schade summarized the effect of the visual pathway as a single shift-invariant “neural image” of the retinal image and measured the contrast threshold for sinusoidal patterns. Wandell reports that Schade described the general shape of the psychophysical linespread function inferred from such measurements as a difference of two Gaussians, the same description used for ganglion cells.

Schade’s single-resolution theory does not predict all the data. In Wandell’s account, mixtures of two gratings are detected at slightly higher contrasts than the theory predicts, and pattern adaptation (staring at a grating lowers sensitivity only near the adapting frequency, as Blakemore and Campbell reported in 1969) suggests several channels tuned to different spatial frequencies: a multiresolution representation. Campbell and Robson (1968) gave a classic test, as Wandell describes it. A square wave of frequency $f$ is a sum of sinusoids at $f$, $3f$, $5f$ and so on, with the fundamental’s amplitude $4/\pi$ times the square wave’s contrast. Above about 1 cpd they found the square wave just detectable when its fundamental reached the sinusoid’s threshold, and distinguishable from a sinusoid only when its third harmonic reached its own threshold, as if the two components were encoded by different channels. Wandell adds that Campbell and Robson’s components were of low contrast and widely separated in frequency, and that such independence is very rare: masking and facilitation are quite common.

The observer’s curve changes with viewing conditions. On bright backgrounds it is band-pass and at low light levels it becomes low-pass, with its peak at lower frequencies (Wandell, ch. 7, from data of Nes and Bouman as he cites them; his text and his figure caption give slightly different peak frequencies). Sensitivity also falls with distance from the fovea. Useful numbers gives the highest detectable spatial frequency as 50–60 cpd at high ambient light levels and 20–30 cpd at low ambient light levels, and a contrast threshold of 1% for a static edge at photopic luminances. The 50–60 cpd limit sits close to the foveal cone Nyquist frequency of about 60 cpd, so in our reading the optics, the mosaic and the neural limit are roughly matched in the fovea. Parametric formulas for the curve exist, among them Mannos and Sakrison’s; none is reproduced here.

Example: a Difference-of-Gaussians Receptive Field

The model in its steady-state spatial form fits in two lines. Write $G_\sigma$ for a two-dimensional Gaussian with standard deviation $\sigma$ and unit volume. A centre–surround receptive field is

$$\mathrm{RF}(x, y) = G_{\sigma_c}(x, y) - w\, G_{\sigma_s}(x, y), \qquad \sigma_s > \sigma_c,$$

where $w$ is the weight of the surround. This parameterization is ours: Wandell describes Gaussian centre and surround mechanisms, with a delayed surround, but gives no formula, and the unit volumes, the weight $w$ and the restriction to static stimuli are choices of this example. A grating of contrast $C$ and spatial frequency $u$ that varies only along $x$ sees the one-dimensional profile, and since a unit-volume Gaussian has the Fourier transform $e^{-2\pi^2\sigma^2u^2}$, the response at a bright bar of the grating is $C\,H(u)$ with

$$H(u) = e^{-2\pi^2\sigma_c^2u^2} - w\, e^{-2\pi^2\sigma_s^2u^2}.$$

Three consequences follow (our derivation). The gain for a uniform field is $H(0) = 1 - w$: a balanced field ($w = 1$) ignores the mean, while an unbalanced one passes part of it. Setting $dH/du = 0$ gives the most effective frequency,

$$u^{*2} = \frac{\ln\!\left(w\,\sigma_s^2/\sigma_c^2\right)}{2\pi^2\left(\sigma_s^2 - \sigma_c^2\right)},$$

which exists when $w\sigma_s^2 > \sigma_c^2$. And because a Gaussian leaves a linear function unchanged, the response to a luminance ramp is $(1 - w)\,c(x)$ wherever the field lies entirely on the ramp, zero for a balanced field, while at a knee, where the slope changes by $\Delta s$, it is

$$R(x_k) = (1 - w)\,c(x_k) + \Delta s\,\frac{\sigma_c - w\,\sigma_s}{\sqrt{2\pi}},$$

since $\int_0^\infty u\,g_\sigma(u)\,du = \sigma/\sqrt{2\pi}$ for a one-dimensional unit-area Gaussian $g_\sigma$. With $\sigma_c < w\sigma_s$ the response dips below the ramp at the lower knee and rises above it at the upper one: a centre–surround model responds where the luminance gradient changes, not where it is constant.

The script below checks all of this on sampled images. Stimuli are contrast patterns $c = (L - L_0)/L_0$ about a mean luminance $L_0$ on a 512 × 512 grid. The grid spacing is 0.5 minutes of arc per pixel, an assumption taken from the foveal L and M cone spacing in Wandell’s Useful numbers, so the grid’s Nyquist frequency is 60 cpd and a frequency $f$ in cpd is $u = f/60$ cycles per minute of arc. The centre has $\sigma_c = 1$ arcmin (2 pixels), also an assumption, not a measured neuron; the surround has $\sigma_s/\sigma_c = 3$ or 6, and $w = 1$ or $0.8$, giving four conditions. Each Gaussian is sampled at the pixels, truncated at $8\sigma$ and normalized to unit sum, which is how scipy.ndimage.gaussian_filter builds its kernels. The script has four parts:

• Gratings. 596 gratings, 0.5 to 60 cpd in steps of 0.1 cpd, of contrast $C = 0.5$ in cosine phase at the centre pixel; the response at the centre divided by $C$ gives the measured transfer function. H1: it agrees with $H$ to $10^{-6}$ relative wherever $|H| > 10^{-3}$. H2: its peak on the 0.1 cpd grid lies within one grid step of $f^* = 60\,u^*$. The column H(f*) prints the closed-form gain at the peak; it decides no criterion.
• Uniform field and checks. C1: the gain for a uniform field equals $1 - w$ within $10^{-12}$. C4: the response computed as a sum over the kernel at the centre pixel equals scipy.ndimage.gaussian_filter’s value at 1, 7 and 30 cpd, within $10^{-12}$.
• Ramp. A ramp from $c = -0.5$ at column 128 to $+0.5$ at column 384, flat outside, filtered with gaussian_filter (mode reflect); its slope is $1/256$ per pixel, $1/128$ per arcmin. H3: the responses at the two knees agree with the closed form to $10^{-3}$ and have opposite signs. C2: on columns 224 to 288, where the largest kernel lies on the ramp, the response equals $(1 - w)\,c$ within $10^{-12}$.
• scikit-image. C3: skimage.filters.difference_of_gaussians on an 8-bit step image (codes 64 and 192), for $w = 1$, equals the hand-computed difference of Gaussians on the image divided by 255 within $10^{-12}$, and 255 times its output equals the hand computation on the raw codes within $10^{-9}$.

The tolerances of H1–H3 come from the batch plan; those of C1–C4 and of the data checks (printed first) are ours, as the plan gave none. All were written down before any code ran.

import os
os.environ["OMP_NUM_THREADS"] = "1"          # one BLAS thread, set before NumPy is imported
os.environ["OPENBLAS_NUM_THREADS"] = "1"
import numpy as np
from scipy import ndimage
from skimage.filters import difference_of_gaussians

PIX = 0.5                     # arcmin per pixel (assumed: foveal L/M cone spacing)
N, X0 = 512, 256              # image size; centre pixel
SC = 1.0                      # centre sigma in arcmin (an assumption)
TR = 8.0                      # truncate each Gaussian at 8 sigma
C = 0.5                       # grating contrast
FREQS = np.round(np.arange(0.5, 60.0 + 1e-9, 0.1), 10)            # cycles per degree
CONDS = [(3, 1.0), (6, 1.0), (3, 0.8), (6, 0.8)]                  # (sigma_s / sigma_c, w)
xx = np.broadcast_to(np.arange(N, dtype=float), (N, N))          # column index; images vary along x only


def taps(sig_px):
    """Sampled Gaussian normalized to unit sum, radius int(TR*sigma + 0.5), as scipy.ndimage builds it."""
    r = int(TR * sig_px + 0.5)
    g = np.exp(-0.5 * np.arange(-r, r + 1) ** 2 / sig_px**2)
    return g / g.sum()


def rf_kernel(ratio, w):
    """2-D centre-minus-surround kernel G(sc) - w G(ss), on the surround's support."""
    gc, gs = taps(SC / PIX), taps(ratio * SC / PIX)
    pad = (gs.size - gc.size) // 2
    gc = np.pad(gc, pad)
    return np.outer(gc, gc) - w * np.outer(gs, gs)


def at_centre(K, img):
    r = K.shape[0] // 2
    return float(np.sum(K * img[X0 - r:X0 + r + 1, X0 - r:X0 + r + 1]))


def H_closed(f, ratio, w):      # f in cpd; sigmas in arcmin; u = f/60 cycles per arcmin
    u = f / 60.0
    return np.exp(-2 * np.pi**2 * SC**2 * u**2) - w * np.exp(-2 * np.pi**2 * (ratio * SC) ** 2 * u**2)


def grating(f):
    return C * np.cos(2 * np.pi * f * (xx - X0) * PIX / 60.0)


def dog_filter(img, ratio, w):
    g = lambda s: ndimage.gaussian_filter(img, s / PIX, mode="reflect", truncate=TR)
    return g(SC) - w * g(ratio * SC)


# Stimuli and data checks
ramp = np.clip(-0.5 + (xx - 128) / 256.0, -0.5, 0.5)                # contrast; knees at x = 128 and 384
step = np.full((N, N), 64, np.uint8)
step[:, X0:] = 192
g7 = grating(7.0)
slope = np.diff(ramp[0])[129:383]
print(f"data errors: grating centre {abs(g7[X0, X0] - C):.1e}, max |c| - C {abs(np.abs(g7).max() - C):.1e}; "
      f"plateaus {np.abs(ramp[:, :129] + 0.5).max():.1e} and {np.abs(ramp[:, 384:] - 0.5).max():.1e}, "
      f"c(128) {abs(ramp[0, 128] + 0.5):.1e}, c(384) {abs(ramp[0, 384] - 0.5):.1e}, slope {np.abs(slope - 1 / 256).max():.1e}; "
      f"step codes {sorted(set(step.ravel().tolist()))}, columns {(step[0] == 64).sum()}/{(step[0] == 192).sum()}")

print(f"\n{'ss/sc':>5} {'w':>4} | {'H1 pts':>6} {'max rel err':>11} | {'f* cpd':>7} {'grid pk':>7} {'|diff|':>6} "
      f"{'H(f*)':>7} | {'C1 DC err':>9} | {'C4 impl':>7}")
for ratio, w in CONDS:
    K = rf_kernel(ratio, w)
    Hn = np.array([at_centre(K, grating(f)) / C for f in FREQS])
    Hc = H_closed(FREQS, ratio, w)
    dom = np.abs(Hc) > 1e-3
    rel = np.max(np.abs(Hn[dom] - Hc[dom]) / np.abs(Hc[dom]))
    fstar = 60 * np.sqrt(np.log(w * ratio**2) / (2 * np.pi**2 * SC**2 * (ratio**2 - 1)))
    fpk = FREQS[np.argmax(Hn)]
    dc = at_centre(K, np.full((N, N), C)) / C
    impl = max(abs(at_centre(K, grating(f)) - dog_filter(grating(f), ratio, w)[X0, X0]) for f in (1.0, 7.0, 30.0))
    print(f"{ratio:5d} {w:4.1f} | {dom.sum():6d} {rel:11.1e} | {fstar:7.4f} {fpk:7.1f} {abs(fpk - fstar):6.4f} "
          f"{H_closed(fstar, ratio, w):7.4f} | {abs(dc - (1 - w)):9.1e} | {impl:7.1e}")

print(f"\n{'ss/sc':>5} {'w':>4} | {'R(128)':>9} {'closed':>9} {'err':>8} | {'R(384)':>9} {'closed':>9} {'err':>8} "
      f"| {'opposite':>8} | {'C2 linear':>9}")
s = 1 / (256 * PIX)                                                 # ramp slope per arcmin
for ratio, w in CONDS:
    R = dog_filter(ramp, ratio, w)
    row = []
    for xk, ds in ((128, s), (384, -s)):
        closed = (1 - w) * ramp[0, xk] + ds * (SC - w * ratio * SC) / np.sqrt(2 * np.pi)
        row += [R[X0, xk], closed, abs(R[X0, xk] - closed)]
    lin = np.abs(R[:, 224:289] - (1 - w) * ramp[:, 224:289]).max()
    print(f"{ratio:5d} {w:4.1f} | {row[0]:+9.6f} {row[1]:+9.6f} {row[2]:8.2e} | {row[3]:+9.6f} {row[4]:+9.6f} "
          f"{row[5]:8.2e} | {str(np.sign(row[0]) == -np.sign(row[3])):>8} | {lin:9.1e}")

print("\nC3 scikit-image difference_of_gaussians (w = 1) on the uint8 step")
for ratio in (3, 6):
    sk = difference_of_gaussians(step, SC / PIX, ratio * SC / PIX, mode="reflect", truncate=TR)
    hand = dog_filter(step / 255.0, ratio, 1.0)
    raw = dog_filter(step.astype(float), ratio, 1.0)
    print(f"  ss/sc = {ratio}: dtype {sk.dtype}, |skimage - hand(img/255)| {np.abs(sk - hand).max():.1e}, "
          f"|255 * skimage - hand(raw codes)| {np.abs(255 * sk - raw).max():.1e}")

The script uses NumPy 2.3.5, SciPy 1.16.3 and scikit-image 0.25.2, uses no random numbers, prints the same output on every run, and runs in about 3 seconds:

data errors: grating centre 0.0e+00, max |c| - C 0.0e+00; plateaus 0.0e+00 and 0.0e+00, c(128) 0.0e+00, c(384) 0.0e+00, slope 0.0e+00; step codes [64, 192], columns 256/256

ss/sc    w | H1 pts max rel err |  f* cpd grid pk |diff|   H(f*) | C1 DC err | C4 impl
    3  1.0 |    350     6.4e-13 |  7.0775     7.1 0.0225  0.6754 |   5.9e-17 | 1.4e-16
    6  1.0 |    350     4.5e-13 |  4.3212     4.3 0.0212  0.8776 |   3.8e-17 | 5.6e-17
    3  0.8 |    350     5.2e-13 |  6.7085     6.7 0.0085  0.6945 |   0.0e+00 | 1.7e-16
    6  0.8 |    350     3.8e-13 |  4.1845     4.2 0.0155  0.8832 |   2.8e-17 | 5.6e-17

ss/sc    w |    R(128)    closed      err |    R(384)    closed      err | opposite | C2 linear
    3  1.0 | -0.006278 -0.006233 4.41e-05 | +0.006278 +0.006233 4.41e-05 |     True |   2.8e-17
    6  1.0 | -0.015639 -0.015584 5.49e-05 | +0.015639 +0.015584 5.49e-05 |     True |   2.8e-17
    3  0.8 | -0.104412 -0.104363 4.84e-05 | +0.104412 +0.104363 4.84e-05 |     True |   2.8e-17
    6  0.8 | -0.111901 -0.111844 5.71e-05 | +0.111901 +0.111844 5.71e-05 |     True |   2.8e-17

C3 scikit-image difference_of_gaussians (w = 1) on the uint8 step
  ss/sc = 3: dtype float64, |skimage - hand(img/255)| 0.0e+00, |255 * skimage - hand(raw codes)| 6.4e-14
  ss/sc = 6: dtype float64, |skimage - hand(img/255)| 0.0e+00, |255 * skimage - hand(raw codes)| 1.0e-13

The checks. All held. The data line shows the stimuli as described, with every error zero: the grating has contrast 0.5 at the centre, the ramp’s plateaus, knee values and slope are exact, and the step has 256 columns at each code (D). The uniform-field gain equals $1 - w$ to within $6 \times 10^{-17}$ (C1), the kernel sum at the centre pixel equals SciPy’s filter to within $2 \times 10^{-16}$ (C4), and the response on the linear part of the ramp equals $(1 - w)\,c$ to $3 \times 10^{-17}$ in all four conditions (C2): a balanced field gives zero on a ramp, to rounding, because each normalized, symmetric kernel reproduces a linear function. scikit-image’s result is identical to the hand computation on the image divided by 255 (C3), which is what its source shows: difference_of_gaussians first converts the image with img_as_float, which maps 8-bit codes to $[0, 1]$, then subtracts the two gaussian outputs. It has no surround weight, so it computes only the $w = 1$ case, and its defaults differ from this script’s: mode='nearest', truncate=4.0, and a high_sigma of 1.6 times low_sigma when none is given, a ratio its docstring attributes to Marr and Hildreth (1980).

The transfer function. H1 held in all four conditions: over the 350 frequencies per condition where $|H| > 10^{-3}$ (0.5 to about 35 cpd), the measured and closed-form transfer functions differ by at most $6.4 \times 10^{-13}$ relative. The sampled Gaussians reproduce the continuous transform almost exactly because, in our reading, a Gaussian 2 or more pixels wide has almost no energy beyond the grid’s Nyquist frequency, and the $8\sigma$ truncation leaves out almost none of its mass. H2 held in all four: no grid peak lies more than 0.0225 cpd from its closed-form peak, well within the 0.1 cpd step. The printed peaks show the model’s geometry. Doubling the surround from $3\sigma_c$ to $6\sigma_c$ moves the most effective frequency down from 7.08 to 4.32 cpd for $w = 1$ and raises the peak gain from 0.68 to 0.88 (0.69 to 0.88 for $w = 0.8$); in our reading, a wider surround cancels only lower frequencies and leaves more of the centre’s response at the peak. Weakening the surround to $w = 0.8$ lowers the peak slightly (6.71 against 7.08 cpd at ratio 3) and leaves a DC gain of 0.2, so that this field is a band-pass filter plus a fraction of the mean.

The ramp. H3 held in all four conditions: no knee response differs from the closed form by more than $5.71 \times 10^{-5}$, below the $10^{-3}$ bar, and the lower knee is negative while the upper knee is positive in every condition. These errors are far larger than the grating errors. In our reading they come from sampling the kernels at whole pixels at a kink of the stimulus, where the sum over pixels and the integral in the closed form differ in a way they do not for a smooth grating; the output shows only their size, equal at both knees by the ramp’s symmetry. For $w = 1$ the knee response is the whole signal: the closed form gives $\mp 0.0062$ for the narrower surround and $\mp 0.0156$ for the wider one, against zero along the ramp. For $w = 0.8$ the knee adds a smaller term to the $(1 - w)\,c = \mp 0.1$ that the field passes from the plateaus: the closed form gives $0.0044$ and $0.0118$ beyond $\mp 0.1$ (our arithmetic from the printed rows). We make no claim here about what an observer sees at the knees of a ramp; the example shows only what the linear model computes.

Process note. The plan for this example listed eleven departures from the batch plan’s description before the pilot: (1) $\sigma_c = 1$ arcmin, which the batch plan left open; (2) truncation at $8\sigma$, because by derivation the default $4\sigma$ would leave out a tail that exceeds H1’s tolerance after renormalization; (3) the 0.1 cpd frequency grid, which sets H2’s grid step; (4) a fixed contrast of 0.5 in cosine phase, with all stimuli in contrast units; (5) the batch plan’s “trapezoidal ramp” built as one ramp between two plateaus, because four knees each outside the 96-pixel surround of the others do not fit in 512 pixels, and in contrast units, so that the signs at the knees for $w = 0.8$ do not depend on a luminance baseline; (6) the step used only as the input of the scikit-image check, not scored as a step response; (7) the zero-response check on the ramp’s linear part extended to $w = 0.8$ as $|R - (1 - w)\,c|$; (8) checks C4 and D and the 255× scaling in C3 added; (9) scikit-image called with mode='reflect' and truncate=8.0 instead of its defaults, so that it computes the same filter as the hand code; (10) grating responses computed as the kernel sum at the centre pixel rather than by filtering 596 full images, which C4 checks; (11) the tolerances of C1–C4 and D set by us. All bars were written before a planning script computed the values a correct implementation reaches, and none was changed afterwards; for C4 and D no reachable value was computed before the pilot, since they check an implementation identity and the generator’s construction. The pilot script also printed the column H(f*), which the plan did not list; the editor decided to keep it as a printed value with no criterion. One change was made between the pilot and the first run. The editor approved a change of print format only: the pilot printed the data checks D as values at 2 to 6 decimals, and the first run was to print them as absolute errors in scientific notation (%.1e), so that they can be decided at the $10^{-12}$ tolerance. The rewritten print also changed what the plateau check measures, which that approval did not cover: the pilot read row 0 only, columns 0 to 127 and 385 to 511, while the first run reads every row and columns 0 to 128 and 384 to 511, so the knee columns 128 and 384 are now included. The data checks held in both runs: the pilot printed every value at its planned value, and the first run prints every error as zero. Every other printed line is identical between the pilot and the first run.

Common Misconceptions

• “The eye works like a camera that sends a picture to the brain.” The eye forms an image like a camera, but the retina transforms it before anything leaves the eye: it reports local contrast rather than light level and splits the signal into parallel streams for different destinations (Wandell, ch. 5). In Wandell’s account, perception is an inference from this transformed signal.
• “Visual resolution is the same everywhere.” Cone density, and with it the sampling rate, is highest in the fovea and falls with eccentricity, so aliasing and the resolution limit start at lower frequencies in the periphery (Wandell, ch. 3), and contrast sensitivity falls too (ch. 7). In dim light the rod-free fovea is less sensitive than the periphery.
• “Contrast sensitivity is one fixed curve.” For neurons and observers alike, the curve is band-pass on bright backgrounds and becomes low-pass in dim light (Wandell, ch. 5 and ch. 7), and it falls with eccentricity. A single published curve describes one set of viewing conditions.
• “Each visual area computes one feature.” The proposals that assign colour, motion or form to separate areas rest, in Wandell’s analysis (ch. 6), on the neuron doctrine, which he finds poorly supported, and he calls the best-known such diagram speculative. Signals of the magnocellular and parvocellular streams also merge within V1.
• “Convolutional networks are models of the visual cortex.” The convolutional network took its local receptive fields and its stacking of simple and complex stages from Hubel and Wiesel’s findings by way of Fukushima’s neocognitron (see receptive field), and Torralba et al. (§1.3.5) count neural networks among the approaches inspired by neuroscience. Inspiration is not identity: the cortex has nonlinear cells, contrast normalization, feedback (most LGN input comes from the cortex) and many specialized areas that the sources above describe and a standard network does not contain.
• “A difference-of-Gaussians filter in a library is the retinal model.” scikit-image’s difference_of_gaussians subtracts two equal-weight Gaussians, so it has no DC response, and its default ratio of 1.6 and its rescaling of integer images (above) are library conventions. A ganglion-cell model needs its own centre size, surround size and weight, and, in Wandell’s account, a delayed surround.

Where It Is Used

• Colour encoding. Three-channel colour imaging rests on trichromacy, the subject of colorimetry and color spaces; the Bayer filter article recounts why its patent samples green twice as densely.
• Perceptual coding. Gamma correction spaces code values so that steps stay near visibility thresholds such as a 1% luminance ratio, and image compression shows how JPEG’s example quantization tables were derived by psychovisual thresholding.
• Quality metrics. Image quality assessment describes metrics built on models of the visual system, and the SSIM paper article describes the error-sensitivity framework SSIM was set against, which weights spatial frequencies by a contrast sensitivity function.
• Centre–surround filters. The Laplacian of Gaussian and its difference-of-Gaussians approximation, which Marr and Hildreth related to cells in the visual system, are the subject of the Laplacian operator, blob detection and scale space articles. Image sharpening relates unsharp masking to a model of early vision whose spectrum resembles the contrast sensitivity function, and tone mapping and low-light enhancement use centre–surround comparisons to set local contrast.
• Oriented and multiscale filters. Gabor filters model V1 simple cells, and image pyramids are multiresolution representations, the kind Wandell’s chapter 7 discusses for human pattern vision.
• Networks and attention. Convolutional networks inherit the receptive field, and Torralba et al. (§1.3.5) count attention modulation among the ideas vision science supplied.
• Sensors. The event camera article traces the sensor’s idea to Mahowald and Mead’s silicon retina.
• Motion. The aperture problem article draws on Wallach’s studies of human motion perception.