Vision Systems & Engineering
NumPy
The Python library of n-dimensional arrays in which Python vision libraries exchange images, with the array model, views and copies, broadcasting, accumulator types and NumPy 2's promotion rules that decide what image arithmetic returns.
beginner
NumPy is the Python library for n-dimensional arrays of fixed-type numbers, together with the element-wise functions, reductions, indexing and random-number generators that operate on them. It is developed by the NumPy Developers, who hold its copyright, and distributed under the 3-clause BSD licence. In computer vision it serves as the common container: OpenCV’s Python functions and scikit-image take and return a digital image as a NumPy array of shape (rows, columns) or (rows, columns, channels), and PyTorch converts such arrays to tensors and back. The NumPy paper is Harris et al. (2020), which the OpenCV article cites for the array model; we have not read it.
This article covers what decides the numbers an image computation returns: how an array is laid out in memory, when indexing shares that memory, how arrays of different shapes are combined, which type a result and its accumulator get, and what changed in NumPy 2. The examples ran with NumPy 2.3.5; newer releases exist (2.5.4 is the latest as of November 2026), and nothing here is claimed about them. Indexing conventions such as image[row, column] belong to the digital image article, OpenCV’s saturating arithmetic and channel order to the OpenCV article, reshaping as a view to linear algebra and lookup tables to point operations.
Purpose
NumPy moves loops over pixels out of Python: an expression on whole arrays is evaluated element by element by universal functions (ufuncs), which the user guide describes as vectorized wrappers around functions of a fixed number of inputs, many of them implemented in compiled C code. Broadcasting, in the guide’s words, is “a means of vectorizing array operations so that looping occurs in C instead of Python”. In our reading, the shared array type is also what lets libraries be combined: one that accepts a NumPy array can take the output of another that produces one.
Core Capabilities
The ndarray. An array is a data buffer plus metadata, which by the copies-and-views page includes the data type (dtype) and the strides, the number of bytes to step in memory to move one position along each axis; the shape gives the size of each axis. NumPy uses C-order indexing, in which the last index usually represents the most rapidly changing memory location, unlike Fortran or IDL (indexing guide). Changing the strides and shape without touching the buffer gives a new way of reading the same data; this is how slicing and transposing work.
Views and copies. “Views are created when elements can be addressed with offsets and strides in the original array”, so basic indexing (integers, slices, ..., None) always creates views, while advanced indexing with integer or boolean arrays always creates copies (copies-and-views page; indexing guide). The base attribute of a view returns the array it views and is None for a copy, and np.shares_memory tests whether two arrays overlap in memory. reshape returns a view where the strides allow it and a copy otherwise, and ravel returns a view wherever possible while flatten always copies.
Broadcasting. When two arrays are combined, NumPy compares their shapes from the trailing dimension leftwards, and two dimensions are compatible when they are equal or one of them is 1; missing dimensions count as size one, and incompatible shapes raise ValueError: operands could not be broadcast together (broadcasting guide). The guide’s own example scales a RGB image by a three-element array, which lines up with the channel axis; in channel-first (CHW) layout the same array lines up with the width.
Reductions and accumulators. The type a reduction accumulates in is not always the input’s. By the sum docstring, the accumulator has the input’s type unless the input is an integer type of less precision than the default platform integer; then a signed input accumulates in the platform integer, and an unsigned input in an unsigned integer of the platform integer’s precision. mean computes in float64 for integer input and in the input’s precision for floating-point input, and its docstring warns that this can be inaccurate for float32: its example mean of values, half 1.0 and half 0.1, returns 0.54999924. Element-wise arithmetic gets no such widening: x * x on a uint16 image is computed in uint16. The diff docstring notes that for unsigned integer arrays “the results will also be unsigned”, so differences of unsigned images wrap; it advises casting to a larger integer type first.
Type promotion. When an operation mixes types, NumPy 2 follows NEP 50: a Python int, float or complex is “weakly” typed and converted to the array’s type, while NumPy scalars and arrays keep their type, and values never decide the result type. A Python integer that the array’s type cannot hold is an error rather than a reason to widen. The NEP also lists the cases where arbitrary Python integers stay allowed, such as comparisons and division, which returns floating point.
Random numbers. The recommended interface is a Generator created with np.random.default_rng; with no seed it draws its state from the operating system and gives different numbers each run, and with a seed it reproduces the sequence. The documentation warns that these generators are designed for statistical modelling and simulation and are not suitable for security or cryptographic purposes.
Minimal Example
The example builds one synthetic 12-bit RGB frame, stored as uint16, and checks five array facts against counts known from its construction. The red channel is below 256 except for a patch with values from 256 to 4095; green and blue have separate value ranges, so the channel means differ.
- Views. A sliced and an advanced-indexed (
np.ix_) crop of one region are edited in place. - Broadcasting. A per-channel mean of shape (3,) is subtracted in HWC layout, in CHW layout and from a CHW strip three pixels wide, against an explicit per-channel loop.
- Accumulators. The local variance in windows, , is computed on the
uint16red channel and on afloat64copy. A square wraps inuint16for , since exceeds 65,535, so exactly the 384 patch pixels wrap and exactly the windows containing one should change. - Promotion. The NEP 50 cases above, on an 8-bit preview (the frame shifted right by 4 bits) and a
float32copy, andastypefrom float to integer. - Frame differencing, the simplest detector in the background subtraction article. A square, 800 levels above the green channel, moves by (3, 5) pixels between two frames with independent integer noise from 0 to 3. is computed in
uint16and inint32and thresholded at 50, against the true motion mask (the pixels the square entered or left).
The hypotheses were written before any script ran: every printed count equals its construction exactly, every dtype and exception matches NEP 50 and the docstrings as read, the frame’s strides are (576, 6, 2) and its CHW transpose’s are (2, 576, 6). A planning script then computed the construction counts from the same data before the plan was frozen. Parts of the example go beyond the batch plan’s description of it, and these departures were approved before the run: the astype truncation check, the sum beside the sum of squares, the base test, the strides and contiguity prints, the noise and the motion-mask score in part 5, the corrected CHW broadcast, and the use of an RGB frame with an 8-bit preview. A pilot of the same script ran first and printed the same output as the run shown, including the failed strides check below; the editor kept the code as run. The script runs in well under a second; two runs printed identical output.
import os
for v in ("OPENBLAS_NUM_THREADS", "OMP_NUM_THREADS", "MKL_NUM_THREADS"):
os.environ[v] = "1" # one BLAS thread, set before NumPy is imported
import numpy as np
from numpy.lib.stride_tricks import sliding_window_view
print("NumPy", np.__version__)
# One synthetic 12-bit RGB frame stored as uint16, shape (rows, cols, channels).
H, W = 64, 96
rng = np.random.default_rng(0)
frame = np.zeros((H, W, 3), np.uint16)
frame[..., 0] = rng.integers(0, 256, (H, W)) # red: dark, below 256 ...
r0, r1, c0, c1 = 20, 36, 40, 64
frame[r0:r1, c0:c1, 0] = rng.integers(256, 4096, (r1 - r0, c1 - c0)) # ... except a bright patch
frame[..., 1] = rng.integers(1000, 2000, (H, W))
frame[..., 2] = rng.integers(2000, 3000, (H, W))
R = frame[..., 0]
means = frame.mean(axis=(0, 1)) # float64 for integer input
gap = min(abs(means[0] - means[1]), abs(means[0] - means[2]), abs(means[1] - means[2]))
print("data:", frame.dtype, frame.shape, "max", frame.max(),
"| red >= 256:", int((R >= 256).sum()), "of patch area", (r1 - r0) * (c1 - c0),
"| outside patch:", int((R >= 256).sum() - (R[r0:r1, c0:c1] >= 256).sum()),
"| channel means %.2f %.2f %.2f, min gap %.2f" % (*means, gap))
# 1. Views and copies.
print("\n1. views")
before = frame.copy()
crop = frame[8:24, 10:40] # basic slicing: a view
fancy = frame[np.ix_(np.arange(8, 24), np.arange(10, 40))] # advanced indexing: a copy
print("shares memory: slice", np.shares_memory(crop, frame), "| fancy", np.shares_memory(fancy, frame))
print("base: slice crop.base is frame", crop.base is frame, "| fancy.base is None", fancy.base is None)
print("strides: frame", frame.strides, "crop", crop.strides,
"| C-contiguous: frame", frame.flags.c_contiguous, "crop", crop.flags.c_contiguous)
fancy += 1
print("pixels changed in frame by editing the fancy crop:", int((frame != before).any(-1).sum()))
crop += 1
print("pixels changed in frame by editing the slice crop:", int((frame != before).any(-1).sum()),
"(crop area", crop.shape[0] * crop.shape[1], ")")
frame[...] = before # undo the edit
# 2. Broadcasting a per-channel mean.
print("\n2. broadcasting")
f = frame.astype(np.float32)
mean = f.mean(axis=(0, 1)) # shape (3,)
loop = np.stack([f[..., c] - mean[c] for c in range(3)], axis=-1)
print("HWC", f.shape, "- mean", mean.shape, "-> wrong values:", int(((f - mean) != loop).sum()))
chw = f.transpose(2, 0, 1) # (3, H, W), a view
print("CHW", chw.shape, "strides", chw.strides)
try:
chw - mean
except ValueError as e:
print("CHW - mean: ValueError:", e)
strip = chw[:, :, :3] # a CHW strip 3 pixels wide
ref = strip - mean[:, None, None]
wrong = (strip - mean) != ref # (3,) lines up with W, not C
print("strip", strip.shape, "- mean: no error, wrong values:", int(wrong.sum()), "of", wrong.size,
"| smallest error %.2f" % np.abs((strip - mean) - ref)[wrong].min())
print("CHW - mean[:, None, None]: wrong values:",
int(((chw - mean[:, None, None]) != loop.transpose(2, 0, 1)).sum()))
# 3. Accumulators: local variance as mean(x*x) - mean(x)^2 in 5x5 windows.
print("\n3. accumulators")
exact = R.astype(np.int64) ** 2
print("squares that wrapped in uint16:", int(((R * R) != exact).sum()))
def local_var(x, k=5):
win = sliding_window_view(x, (k, k)) # (H-k+1, W-k+1, k, k), a view
return (win * win).mean(axis=(-2, -1)) - win.mean(axis=(-2, -1)) ** 2
v16, v64 = local_var(R), local_var(R.astype(np.float64))
diff = np.abs(v16 - v64)
hit = diff != 0
print("windows whose variance changed:", int(hit.sum()), "of", hit.size,
"| construction (16+4) x (24+4) =", (r1 - r0 + 4) * (c1 - c0 + 4),
"| smallest change %.1f, largest elsewhere %.3g" % (diff[hit].min(), diff[~hit].max()))
tot = R.sum()
print("R.sum():", tot, tot.dtype, "| exact", int(R.astype(np.int64).sum()),
"|| (R*R).sum():", (R * R).sum(), "| exact", int(exact.sum()))
# 4. NEP 50 promotion: Python scalars are weak, NumPy scalars are not.
print("\n4. promotion")
u8 = (frame >> 4).astype(np.uint8) # 8-bit preview of the 12-bit frame
print("u8 + 1 ->", (u8 + 1).dtype, "| u8 + np.int64(1) ->", (u8 + np.int64(1)).dtype,
"| f32 * 0.5 ->", (f * 0.5).dtype)
try:
u8 + 300
except Exception as e:
print("u8 + 300 ->", type(e).__name__ + ":", e)
x = np.array([2.7, -2.7, 0.5, -0.5])
print("astype(int16) of", x, "->", x.astype(np.int16),
"| equals np.trunc:", bool((x.astype(np.int16) == np.trunc(x)).all()))
# 5. Frame differencing on a moving square.
print("\n5. differencing")
G = frame[..., 1].astype(np.int64)
s, (ya, xa), (dy, dx), T = 12, (20, 30), (3, 5), 50
sq_a = np.zeros((H, W), bool); sq_a[ya:ya + s, xa:xa + s] = True
sq_b = np.zeros((H, W), bool); sq_b[ya + dy:ya + dy + s, xa + dx:xa + dx + s] = True
a = (G + 800 * sq_a + rng.integers(0, 4, (H, W))).astype(np.uint16)
b = (G + 800 * sq_b + rng.integers(0, 4, (H, W))).astype(np.uint16)
truth = sq_a ^ sq_b # pixels the square entered or left
d16 = np.abs(a - b) # a - b wraps where b > a
d32 = np.abs(a.astype(np.int32) - b.astype(np.int32))
print("max a, b:", a.max(), b.max(), "| true motion pixels:", int(truth.sum()))
print("uint16 |a - b| differs from int32 at", int((d16 != d32).sum()), "pixels | b > a at",
int((b > a).sum()))
for name, d in (("int32 ", d32), ("uint16", d16)):
mask = d > T
print(name, "mask: false positives", int((mask & ~truth).sum()),
"| false negatives", int((~mask & truth).sum()))
print("pixels outside the true motion with b > a:", int(((b > a) & ~truth).sum()))
NumPy 2.3.5
data: uint16 (64, 96, 3) max 4090 | red >= 256: 384 of patch area 384 | outside patch: 0 | channel means 253.85 1501.21 2496.79, min gap 995.57
1. views
shares memory: slice True | fancy False
base: slice crop.base is frame True | fancy.base is None True
strides: frame (576, 6, 2) crop (576, 6, 2) | C-contiguous: frame True crop False
pixels changed in frame by editing the fancy crop: 0
pixels changed in frame by editing the slice crop: 480 (crop area 480 )
2. broadcasting
HWC (64, 96, 3) - mean (3,) -> wrong values: 0
CHW (3, 64, 96) strides (4, 1152, 12)
CHW - mean: ValueError: operands could not be broadcast together with shapes (3,64,96) (3,)
strip (3, 64, 3) - mean: no error, wrong values: 384 of 576 | smallest error 995.57
CHW - mean[:, None, None]: wrong values: 0
3. accumulators
squares that wrapped in uint16: 384
windows whose variance changed: 560 of 5520 | construction (16+4) x (24+4) = 560 | smallest change 44564.5, largest elsewhere 0
R.sum(): 1559679 uint64 | exact 1559679 || (R*R).sum(): 138166405 | exact 2345091205
4. promotion
u8 + 1 -> uint8 | u8 + np.int64(1) -> int64 | f32 * 0.5 -> float32
u8 + 300 -> OverflowError: Python integer 300 out of bounds for uint8
astype(int16) of [ 2.7 -2.7 0.5 -0.5] -> [ 2 -2 0 0] | equals np.trunc: True
5. differencing
max a, b: 2794 2795 | true motion pixels: 162
uint16 |a - b| differs from int32 at 2353 pixels | b > a at 2353
int32 mask: false positives 0 | false negatives 0
uint16 mask: false positives 2272 | false negatives 0
pixels outside the true motion with b > a: 2272
The data checks held: the frame’s maximum is 4090, all 384 red values of 256 or more lie in the patch, the channel means differ by at least 995.57, and both frames of part 5 stay below 4096.
Views. Editing the sliced crop changed exactly its 480 pixels in the frame, and editing the advanced-indexed crop changed none; base tells them apart as the copies page says, and np.shares_memory agrees. The frame’s strides are the hypothesised (576, 6, 2); the crop keeps them, so its rows are not adjacent in memory and it is not C-contiguous. These checks held.
Broadcasting. In HWC layout the mean lines up with the channel axis and every value is right. The full CHW array raises the documented error. The strip, whose width is 3, raises nothing and is wrong in 384 of its 576 values: in each row the mean of channel was subtracted from channel at column , which is right only where , three of the nine channel-column pairs. The smallest error is the smallest gap between two channel means. Reshaping the mean to (3, 1, 1) puts it on the channel axis, and the full CHW array then has no wrong value. These checks held. The strides check of this part failed: the CHW array printed strides (4, 1152, 12), where the hypothesis was (2, 576, 6).
Accumulators. The uint16 squares wrapped at the 384 patch pixels and nowhere else, and the variance changed in exactly the 560 windows the construction predicts, by at least 44,564.5 squared levels, while every other window matched the float64 result exactly, since squares below 65,536 are exact in uint16. The plain sum came back as uint64 and exact, as the sum docstring describes for unsigned input; the sum of the wrapped squares was already wrong before it reached the accumulator. These checks held.
Promotion. All four NEP 50 cases held: Python scalars kept the array’s type, np.int64(1) promoted the uint8 array to int64, and 300 raised an OverflowError. NEP 50 itself names the error differently: its table of changed behaviours gives array([1], uint8) + 300 as an “Exception”, and a prose example says np.arange(10, dtype=np.int8) + 256 “raises a TypeError”. astype truncated toward zero, like np.trunc, so rounding needs np.rint first; this check held.
Frame differencing. In uint16, wraps to wherever , and the absolute value of an unsigned number is itself, so the two results differed at exactly the 2,353 pixels where . Every wrapped value is far above the threshold, so each pixel where noise made brighter than , by one to three levels, became motion: the uint16 mask had 2,272 false positives, exactly the pixels outside the true motion with , and no misses, while the int32 mask was exact. These checks held. np.diff along the time axis of an unsigned video wraps in the same way.
Ecosystem and Integrations
OpenCV’s cv2 functions take and return NumPy arrays, and its article shows where its conventions differ from NumPy’s: channel order, saturation instead of wrapping, and negative-stride views it refuses as outputs. PyTorch exchanges data with NumPy through torch.from_numpy() and .numpy() without copying, as the PyTorch article describes, so a tensor made this way shares memory with the array, as a view does. scikit-image’s user guide states that images in scikit-image are represented by NumPy ndarrays, and many examples on this site combine NumPy with SciPy and scikit-image. The mixed precision training article uses NumPy to emulate 16-bit and 8-bit floating-point formats, and the pixel article compares the resampling conventions of SciPy and scikit-image on NumPy arrays.
Alternatives
Other array libraries fill a similar space with their own types and rules. NEP 50’s related-work section names JAX, which also treats Python scalars as weak, applies the rule in most functions as well, and records the weak type on the array (jnp.array(1) stays weakly typed), and data-apis.org, which suggests the same rule for Python scalars; neither was read here. In deep-learning code the tensors of PyTorch take the array’s place, and they exchange memory with NumPy as described above.
Versions and Compatibility
NumPy 2.0 changed several behaviours that can alter the results of older image code. The migration guide describes them:
- Promotion follows NEP 50, and “the precision of scalars is now preserved consistently”:
np.float32(3) + 3.now returns afloat32, and afloat32array plusnp.float64(3)now returns afloat64array. The NEP traces the old, value-based rules back to NumPy 1.7 and notes that 1.24 offered a preliminary, opt-in implementation through theNPY_PROMOTION_STATEenvironment variable. Its backwards-compatibility section singles out a value taken from auint8storage array and multiplied by 100, which used to be computed in a wider integer and now stays inuint8and can overflow. - The default integer “is now 64bit on all 64bit systems” and equivalent to
np.intp; it was previously equivalent to the Clongtype, and the guide’s section on it is headed “Windows default integer”. - The
copykeyword. In NumPy 2.3.5,np.array(..., copy=False)andnp.asarray(..., copy=False)raise aValueError“if a copy cannot be avoided”, andasarray’s defaultcopy=Nonecopies only if needed (docstrings;asarraygained the keyword in 2.0.0). The guide advisesnp.asarray(...)in place of oldnp.array(..., copy=False)calls.ndarray.astypeis not among the functions the guide lists: by its docstring, withcopy=Falseit returns the input array only if the dtype, order and subclass requirements are satisfied, and a new array otherwise.
In our reading, code written for NumPy 1.x that mixes NumPy scalars with small integer images, or passes copy=False, needs re-checking under 2.x. The GitHub releases list 2.4.6 on 19 May 2026, 2.5.0 on 21 June 2026 and 2.5.4 on 10 October 2026, the latest as of November 2026; this article’s examples ran with 2.3.5.
Licence. NumPy is distributed under the 3-clause BSD licence. Its LICENSES_bundled.txt lists the compatibly licensed code it includes: LAPACK-lite (BSD-3), Dragon4 (MIT) and libdivide (Zlib), bundled in the repository and source distributions, and Meson (Apache 2.0), spin (BSD-3) and tempita (MIT), vendored there but not installed in built packages.
Limitations
- Integers wrap silently. Fixed-width array arithmetic keeps its type and wraps on overflow without an error, as parts 3 and 5 of the example show (NEP 50 notes that NumPy usually warns for the scalar operators); the data-types guide shows
np.power(100, 9)giving in 32-bit integers. Converting to a wider or floating-point type before squaring or subtracting avoids the wrap, as the same parts show. - Nothing in an array records what its axes mean. Channel order, HWC against CHW, and whether an array is a view are not checked, so a coincidence of sizes gives wrong values rather than an error, as part 2 shows.
- Precision follows the input. Reductions over
float32images run infloat32, and under NEP 50 a Python float does not widen them. - The random generators are not for security; the documentation points to Python’s
secretsmodule for cryptographic use.
Related
- OpenCV
The open-source C++ library of computer vision and image processing functions, with Python bindings that work on NumPy arrays; its modules, conventions, versions from 1.0 to 5.0, contrib modules and licence.
- PyTorch
An open-source Python library for building and training neural networks with tensors, automatic differentiation and GPU acceleration, run eagerly as ordinary Python code and optionally compiled or exported for deployment.
- Point Operations
Operations that compute each output pixel from the input value at the same pixel alone, such as negatives, log and power-law maps, contrast stretches, windows and thresholds; why every one is a look-up table, and what an 8-bit table merges, skips and cannot invert.
- Pixel
The pixel as picture element, where the word came from, the different things it names in images, sensors and displays, how pixels are counted, and why the position of a pixel's centre changes every resize.
- Mixed-Precision Training
Training a network with most of its arithmetic and storage in 16-bit (FP16 or BF16) or 8-bit floating point while keeping the numerically sensitive parts in FP32, using an FP32 master copy of the weights, loss scaling, FP32 accumulation and per-tensor scales for FP8.
- Background Subtraction
Separating moving objects from a static camera's scene by comparing each frame with a model of the background, from frame differencing and running averages to per-pixel Gaussian mixtures, kernel density estimates, ViBe and trained networks, with the problems that break them and the CDnet benchmark that measures them.
References
- NumPy Developers (2025). Copies and views, doc/source/user/basics.copies.rst, NumPy v2.3.5 source code.
- NumPy Developers (2025). Indexing on ndarrays, doc/source/user/basics.indexing.rst, NumPy v2.3.5 source code.
- NumPy Developers (2025). Broadcasting, doc/source/user/basics.broadcasting.rst, NumPy v2.3.5 source code.
- NumPy Developers (2025). Data types, doc/source/user/basics.types.rst, NumPy v2.3.5 source code.
- NumPy Developers (2025). Universal functions (ufunc) basics, doc/source/user/basics.ufuncs.rst, NumPy v2.3.5 source code.
- NumPy Developers (2025). Random sampling, doc/source/reference/random/index.rst, NumPy v2.3.5 source code.
- Berg, S. NEP 50 — Promotion rules for Python scalars (status Final), doc/neps/nep-0050-scalar-promotion.rst, NumPy v2.3.5 source code.
- NumPy Developers (2025). NumPy 2.0 migration guide, doc/source/numpy_2_0_migration_guide.rst, NumPy v2.3.5 source code.
- NumPy Developers (2025). Docstrings of sum and mean, numpy/_core/fromnumeric.py, NumPy v2.3.5 source code.
- NumPy Developers (2025). Docstrings of array, asarray and ndarray.astype, numpy/_core/_add_newdocs.py, NumPy v2.3.5 source code.
- NumPy Developers (2025). Docstring of diff, numpy/lib/_function_base_impl.py, NumPy v2.3.5 source code.
- NumPy Developers (2025). LICENSE.txt (BSD 3-clause) and LICENSES_bundled.txt, NumPy v2.3.5 source code.
- NumPy (2026). NumPy 2.5.4 release, GitHub release page, 10 October 2026.
- scikit-image team (2025). A crash course on NumPy for images, doc/source/user_guide/numpy_images.rst, scikit-image v0.25.2 source code.
- Harris, C. R., Millman, K. J., van der Walt, S. J., Gommers, R., Virtanen, P., Cournapeau, D., Wieser, E., Taylor, J., Berg, S., Smith, N. J., et al. (2020). Array programming with NumPy. Nature, 585(7825), 357–362. (not read)