Vision Systems & Engineering
Image File Formats
What PNG, TIFF, JPEG files, OpenEXR and RAW containers store, including chunks, tags, sample types, bit depths and metadata; how far each one's dynamic range reaches; and what Pillow, imageio and tifffile return when they read them.
intermediate
An image file is a container: a byte layout that records an array’s size, sample type, channels and colour meaning, holds the pixel data in one or more blocks, and carries metadata next to it. A codec is the method that turns pixels into a compressed bitstream inside that container. Image Compression covers the codecs (JPEG’s DCT pipeline, JPEG 2000, lossless coding), and Digital Image puts lossless and lossy formats side by side in outline. This article covers the containers vision code reads and writes every day: PNG, TIFF and BigTIFF, JPEG files, OpenEXR and camera RAW. It shows what each can hold and what three Python libraries (Pillow 12.0.0, imageio 2.37.2 and tifffile 2025.10.4) hand back when they read one.
For vision work the question is usually not size but fidelity. Can the container hold the samples a sensor or renderer produced (12- or 16-bit integers, linear light above 1.0, floating-point depth), and will the library that reads it return the same array? The example writes one ramp across 30 stops of linear light and one 12-bit pattern into five containers, parses a hand-written PNG byte by byte, and measures what comes back.
Purpose
Files are where a vision pipeline’s data stop being arrays. A camera service saves frames, a dataset ships thousands of images, a renderer writes linear radiance with depth beside it, and a slide scanner writes gigapixel pyramids. Each choice of container sets:
- The sample type. 8 or 16-bit unsigned integers, signed integers, or 16 or 32-bit floats.
- The layout. Interleaved or planar channels, rows, strips or tiles, one image or many.
- The colour meaning. Whether values are proportional to light or gamma-encoded, and in which primaries.
- The metadata. Capture settings, orientation, scale factors and text.
- The compression. Whether it is lossless, lossy, or a choice.
Several published articles own one of these cases. Digital Camera owns RAW, DNG and the history of Exif. Model Deployment owns the divergence between decoders that do and do not apply the Exif orientation tag. Whole-Slide Imaging owns pyramidal TIFF, BigTIFF and OME-TIFF slides. Depth Map owns 16-bit depth PNGs with a fixed scale, and HDR Imaging owns floating-point radiance maps. This article gives the containers themselves.
Core Capabilities
PNG
PNG is defined by the W3C’s PNG Specification, whose Third Edition became a W3C Recommendation on 24 June 2025 and is the latest edition as of November 2026. A PNG datastream is an 8-byte signature, 89 50 4E 47 0D 0A 1A 0A, followed by chunks that begin with IHDR and end with IEND (§5.1–§5.2). Each chunk has three or four fields, since the data field may be empty (§5.3):
- a 4-byte length that counts only the data;
- a 4-byte type whose bytes are ISO 646 letters;
- the data;
- a 4-byte CRC computed over the type and the data, but not the length.
The CRC is the 32-bit CRC that GZIP also uses. It is initialized to all ones, processes each byte from its least significant bit, and is inverted at the end (§5.5). The case of each letter in a chunk type carries a property. An uppercase first letter, as in IHDR, IDAT and IEND, marks a critical chunk, and a lowercase one, as in sBIT or gAMA, an ancillary chunk that a decoder may ignore (§5.4).
IHDR holds width, height, bit depth, colour type, and the compression, filter and interlace methods (§11.2.1). Colour type is the sum of 1 (palette), 2 (truecolour) and 4 (alpha). Table 12 restricts the bit depths each type allows:
| Colour type | Image type | Bit depths |
|---|---|---|
| 0 | Greyscale | 1, 2, 4, 8, 16 |
| 2 | Truecolour (RGB) | 8, 16 |
| 3 | Indexed colour | 1, 2, 4, 8 |
| 4 | Greyscale with alpha | 8, 16 |
| 6 | Truecolour with alpha | 8, 16 |
So PNG stores integers only: up to 16 bits per sample, 48 bits for RGB and 64 for RGBA. Multi-byte values, 16-bit samples included, are in network byte order, most significant byte first (§7.1–§7.2). Before compression each scanline gets a filter-type byte and is filtered with one of five filter types; type 0, None, leaves the bytes unchanged, and the other four subtract a function of neighbouring bytes (§7.3, §9). The filtered scanlines are then compressed with deflate in the zlib format, the only compression method the specification defines (§10.1). PNG has no lossy mode. The losses that happen on the way to a PNG come from what was put into it: rounding to 8 bits, clipping, or a palette.
Sample depths that PNG does not allow, such as 12 bits, are scaled up to the next allowed depth (§4.4.5, §12.4). The specification gives the linear formula and left bit replication, often faster to compute, which shifts the bits up and repeats the top bits in the gap; it says the two never differ by more than one. An sBIT chunk records the original number of significant bits, and a decoder recovers the original samples by shifting right (§11.3.2.4, §13.12).
Ancillary chunks carry colour and other metadata:
- gAMA, cHRM, sRGB and iCCP give the transfer function, the primaries and white point, the sRGB space, or an embedded ICC profile (§11.3.2). Without them or cICP, RGB samples are “uncalibrated device-dependent color” (§6.1), and the specification notes that sample values are not necessarily proportional to light intensity. Gamma Correction explains what such an encoding does to the steps an 8-bit file can represent.
- cICP, mDCV and cLLI are new in the Third Edition. cICP identifies primaries, transfer function and matrix coefficients by ITU-T H.273 code points, and mDCV and cLLI carry mastering-display and content-light-level metadata for HDR content, typically with the PQ transfer function of BT.2100 (§11.3.2.6–§11.3.2.8). PNG can therefore signal HDR content, although its samples remain integers.
- eXIf carries an Exif profile, at most one per datastream. The specification recommends that decoders without independent knowledge of its validity treat it as of “historical value only”, since the image may have been edited after the profile was written (§11.3.4.5).
- APNG, frame-based animation in the acTL, fcTL and fdAT chunks, is part of the Third Edition. A decoder that does not know these chunks shows the static image (§4.9).
TIFF and BigTIFF
TIFF Revision 6.0, published by Aldus on 3 June 1992, is a tagged container. An 8-byte header gives the byte order, “II” for little-endian or “MM” for big-endian, the number 42, and the offset of the first image file directory (IFD) (p. 13). An IFD is a count followed by 12-byte entries, each a tag, a field type, a count and a value or an offset to it, and ends with the offset of the next IFD or 0 (pp. 14–15). Chained IFDs give multipage files. Pixel data sit in strips (RowsPerStrip, StripOffsets) or, for large images, in tiles of equal size, padded at the borders (Section 15).
The tags that decide what a sample means are:
- BitsPerSample, the number of bits per sample;
- SampleFormat (tag 339), with values 1 for unsigned integers (the default), 2 for two’s complement signed integers, 3 for IEEE floating point and 4 for undefined. The field “does not specify the size of data samples”; BitsPerSample does (Section 19, p. 80);
- PhotometricInterpretation, for example BlackIsZero, WhiteIsZero, RGB or palette;
- Compression (tag 259). Baseline TIFF has 1 (none), 2 (CCITT modified Huffman, bilevel only) and 32773 (PackBits, a byte-oriented run-length scheme) (pp. 17–18, 30–31). The extensions add, among others, LZW as scheme 5 (Section 13) and JPEG (Section 22). tifffile’s own table also lists 8, ADOBE_DEFLATE (“Deflate, aka ZLIB”), a value that TIFF 6.0 does not list;
- Orientation (tag 274), which says which visual side the 0th row and column represent. The default is 1, and orientations other than 1 are not a Baseline requirement (pp. 36–37).
TIFF 6.0 states that “the largest possible TIFF file is 2**32 bytes in length” (p. 13), and its header and directory entries hold offsets in 4 bytes (pp. 13–15). BigTIFF lifts the limit. The Library of Congress describes it as using 64-bit offsets, marked by version 43 instead of 42 in the header, with 8-byte offsets and 20-byte directory entries against TIFF’s 12. It keeps TIFF 6’s tags. Whole-Slide Imaging describes the vendor slide formats built on tiled pyramidal TIFF and BigTIFF, and DNG, as Digital Camera reports, is a public extension of TIFF 6.0.
In our reading, the breadth that makes TIFF useful for scientific data is also its interoperability risk. A TIFF can hold float samples, signed integers, any number of channels, tiles and pages, but each reader supports a subset, and Baseline readers are not required to handle extensions (p. 12).
JPEG files: JFIF and Exif
The Library of Congress describes JFIF as a minimal format that lets JPEG bitstreams be exchanged between platforms and applications, and whose one requirement beyond the JPEG interchange syntax is an APP0 marker segment right after the start-of-image marker. JFIF 1.02 dates from 1992, and essentially the same specification was published as ITU-T T.871 in 2011 and ISO/IEC 10918-5 in 2013. JPEG files with Exif metadata use baseline JPEG and carry the metadata in APP1 and APP2 segments, and the Library of Congress cites several websites as reporting that, because of where those segments sit, a valid Exif JPEG cannot also be a valid JFIF file. The baseline process works on 8-bit samples, as Image Compression describes; that article covers T.81’s other processes, including lossless coding. What Exif records, and why orientation tags make decoders disagree, is described in Digital Camera and Model Deployment.
OpenEXR
OpenEXR, which the Library of Congress describes as developed by Industrial Light & Magic in 1999 and a project of the Academy Software Foundation since 2019, stores scene-linear images in floating point. Its technical introduction (v3.5.4) lists three channel types: HALF (16-bit floats, for regular image data), FLOAT (32-bit IEEE-754, where HALF’s range or precision is not enough, for example depth) and UINT (32-bit unsigned integers, for example object identifiers). A file can hold any set of named channels with their own sampling rates, scanline or tiled storage, multi-resolution levels, several parts and, since OpenEXR 2.0, deep data with a varying number of samples per pixel.
HALF has 1 sign bit, 5 exponent bits and 10 mantissa bits. The introduction gives its range as roughly to , with numbers below denormalized. The normalized range covers 30 stops “with 1024 steps per stop”, and the denormalized numbers add another ten stops with decreasing precision. The values are linear in scene light, with 0.18 as middle grey and 1.0 as no clamping limit. Most of the compression methods are lossless: PIZ, ZIP, ZIPS, RLE, ZSTD, HTJ2K256 and HTJ2K32. PXR24, B44, B44A, DWAA, DWAB and LJ2K are lossy. PXR24 rounds 32-bit floats to 24 bits and keeps HALF and UINT exact. HDR Imaging advises keeping a radiance map in floating point, and the Quantization article tabulates the FP16 format’s range and machine epsilon.
RAW and DNG
Camera RAW files store the sensor’s mosaic before demosaicing and most processing, 10 to 14 bits per value in the figures Digital Camera reports. That article describes manufacturers’ raw formats as proprietary and varying between cameras, and DNG, the public extension of TIFF 6.0, as the answer, with tags for black and white levels, colour matrices and a noise profile.
Bit depth and range at a glance
| Container | Sample types | Largest integer depth | Floats |
|---|---|---|---|
| PNG | unsigned integers | 16 bits per sample | none |
| TIFF 6.0 | unsigned and signed integers, IEEE floats (SampleFormat) | set by BitsPerSample | yes |
| JPEG (baseline, in JFIF or Exif) | 8-bit samples | 8 | none |
| OpenEXR | HALF, FLOAT, UINT | 32 (UINT) | 16 and 32-bit |
The example measures what the integer and float rows of this table mean for linear light.
Minimal Example
The script writes two synthetic images with exact ground truth into in-memory files and reads them back. Nothing is written to disk.
- A radiance ramp. Linear-light values for , stored as float32: 30 stops from to just below , 64 samples per stop, repeated over 4 rows. The float32 array is the ground truth.
- A 12-bit pattern. A RGB image whose red channel holds every 12-bit code once (), with and an checkerboard of 0 and 4095. It is scaled up to 16 bits, as PNG’s §12.4 requires for depths it does not support, here by left bit replication, .
The containers are:
- (a) an 8-bit greyscale PNG of , written by Pillow;
- (a2) a 16-bit greyscale PNG of , written by Pillow as mode
I;16; - (b) a 16-bit greyscale PNG of the pattern’s red channel, written by Pillow;
- (c) a 16-bit RGB PNG of the pattern, written by hand with
structandzlib. It has the signature, IHDR with colour type 2 and depth 16, an sBIT chunk recording 12 bits, one IDAT with filter type 0 on every row, and IEND. The file is then parsed back by hand, with each CRC verified by a CRC written from §5.5 rather than taken fromzlib, and finally opened with Pillow. It is written by hand because Pillow 12.0.0 cannot write 16-bit RGB PNG: its table of output modes (_OUTMODESin PngImagePlugin.py) has no such entry; - (d) a float16 TIFF and (e) a float32 TIFF of the ramp, written by tifffile with
compression="zlib". Without the optional imagecodecs package, tifffile compresses with its own fallback around Python’szlib. float16 stands in for OpenEXR’s HALF, which cannot be written here: OpenEXR’s Python bindings are not installed, and imageio’s first-choice EXR plugin uses the FreeImage library, which imageio downloads separately, so it is not called.
The measure for the ramp is the maximum relative error in each stop , over its samples. The integer files are decoded as or . Before the run we wrote down these predictions:
- H1. (b), (c) as parsed by hand, and (e) round-trip exactly.
- H2. float16’s error is at most in every normal stop ( to 9) and above in every subnormal stop ( to ). Rounding to nearest with 10 mantissa bits allows at most half a step, of the value, in the normal range; below the step stays fixed at .
- H3. Pillow decodes the 48-bit PNG to 8-bit RGB. Every sample equals , because Pillow’s unpacker for 16-bit big-endian RGB (
unpackRGB16Bin Unpack.c) copies the first byte of each sample, its high byte. - H4. For the 8-bit PNG: error exactly 1 (stored as zero) in stops to , where every sample is below (in stop , its lowest samples); at most the half-step bound in stops to ; and in stops 0 to 9, where the value is clipped at 1.
- H5. For the 16-bit PNG: error 1 in stops to , at most in stops to , and clipping as in H4 above 1. float16 has the smaller maximum error in every stop up to , and the 16-bit integer file in every stop from to .
The checks C1–C8 confirm the data, the hand CRC against zlib.crc32, the parsed header, §12.4’s statement that left bit replication is never more than one away from the linear formula, sBIT recovery, the TIFF tags, and that the files holding lossy sample types return exactly the 8-bit, 16-bit and float16 arrays written to them. It uses NumPy 2.3.5, Pillow 12.0.0 and tifffile 2025.10.4, has no random numbers, and runs in about a tenth of a second. Two runs printed identical output.
import os
os.environ["OPENBLAS_NUM_THREADS"] = "1" # one BLAS thread, before NumPy is imported
import io, struct, zlib
import numpy as np
import tifffile
from PIL import Image
# ---- Synthetic data with exact ground truth -------------------------------------------
# A linear-light ramp: 64 samples per stop from 2^-20 to just below 2^10 (30 stops), 4 rows.
k = np.arange(1920)
x = np.tile((2.0 ** (-20 + k / 64)).astype(np.float32), (4, 1)) # ground truth, float32
stop = -20 + k // 64
# A 12-bit pattern, 64 x 64 RGB: R holds every code 0..4095 once, G = 4095 - R, B a checkerboard.
r, c = np.mgrid[0:64, 0:64]
R = 64 * r + c
p12 = np.stack([R, 4095 - R, 4095 * (((r // 8) + (c // 8)) % 2)], axis=-1).astype(np.uint16)
# 12 -> 16 bits by left bit replication (PNG 3rd ed., 12.4); sBIT will record 12 bits.
p16 = (p12 << 4) | (p12 >> 8)
print("C1 data: ramp strictly increasing", bool(np.all(np.diff(x[0].astype(np.float64)) > 0)),
"| distinct", np.unique(x[0]).size, "| first == 2^-20", bool(x[0, 0] == 2.0 ** -20),
"| last < 2^10", bool(x[0, -1] < 2.0 ** 10), "| samples per stop", sorted(set(np.bincount(stop + 20).tolist())))
print(" pattern: R codes = 0..4095 each once", bool(np.array_equal(np.sort(p12[..., 0].ravel()), np.arange(4096))),
"| channel maxima", p12.reshape(-1, 3).max(0).tolist())
# ---- (c) a 16-bit RGB PNG written by hand ---------------------------------------------
def crc_by_hand(data): # PNG 5.5: init all ones, LSB first, invert at the end
crc = 0xFFFFFFFF
for byte in data:
crc ^= byte
for _ in range(8):
crc = (crc >> 1) ^ (0xEDB88320 if crc & 1 else 0) # x^32+x^26+...+1, bit-reversed
return crc ^ 0xFFFFFFFF
def chunk(ctype, data): # length, type, data, CRC over type + data
return struct.pack(">I", len(data)) + ctype + data + struct.pack(">I", zlib.crc32(ctype + data))
h, w, _ = p16.shape
ihdr = struct.pack(">IIBBBBB", w, h, 16, 2, 0, 0, 0) # depth 16, colour type 2, no interlace
rows = p16.astype(">u2").reshape(h, w * 3) # big-endian samples, R G B per pixel
raw = b"".join(b"\x00" + row.tobytes() for row in rows) # filter type 0 (None) on every row
chunks = [(b"IHDR", ihdr), (b"sBIT", bytes([12, 12, 12])), (b"IDAT", zlib.compress(raw, 9)), (b"IEND", b"")]
png48 = b"\x89PNG\r\n\x1a\n" + b"".join(chunk(t, d) for t, d in chunks)
print("C2 CRC by hand vs zlib.crc32, mismatches:",
sum(crc_by_hand(t + d) != zlib.crc32(t + d) for t, d in chunks), "of", len(chunks))
print(" first 33 bytes:", png48[:33].hex(" "))
# ---- parse it back by hand ------------------------------------------------------------
assert png48[:8] == b"\x89PNG\r\n\x1a\n"
pos, parsed, bad_crc = 8, {}, 0
while pos < len(png48):
n, = struct.unpack(">I", png48[pos:pos + 4]); ctype = png48[pos + 4:pos + 8]
data = png48[pos + 8:pos + 8 + n]; crc, = struct.unpack(">I", png48[pos + 8 + n:pos + 12 + n])
ok = crc_by_hand(ctype + data) == crc
bad_crc += not ok
print(f" chunk {ctype.decode()} length {n:6d} CRC {crc:08x} verified {ok}")
parsed[ctype] = parsed.get(ctype, b"") + data
pos += 12 + n
W, H, depth, ctype_, comp, filt, inter = struct.unpack(">IIBBBBB", parsed[b"IHDR"])
print("C3 CRC failures:", bad_crc)
print("C6 IHDR: width", W, "height", H, "depth", depth, "colour type", ctype_, "compression", comp,
"filter", filt, "interlace", inter, "| sBIT", list(parsed[b"sBIT"]))
stream = np.frombuffer(zlib.decompress(parsed[b"IDAT"]), np.uint8).reshape(H, 1 + W * 3 * 2)
print("C3 filter bytes all 0:", bool(np.all(stream[:, 0] == 0)))
back = stream[:, 1:].copy().view(">u2").reshape(H, W, 3).astype(np.uint16)
# ---- (b) 16-bit greyscale PNG with Pillow (I;16), and the 12-bit originals via sBIT -----
def png_roundtrip(a):
buf = io.BytesIO(); Image.fromarray(a).save(buf, "PNG"); buf.seek(0)
im = Image.open(buf); return im.mode, np.asarray(im)
v_lin = np.floor(np.arange(4096) * 65535 / 4095 + 0.5).astype(np.int64)
v_lbr = (np.arange(4096) << 4) | (np.arange(4096) >> 8)
print("C4 left bit replication vs linear (12.4): max |diff|", int(np.abs(v_lbr - v_lin).max()),
"| codes differing", int((v_lbr != v_lin).sum()), "of 4096")
mode_b, grey16 = png_roundtrip(p16[..., 0])
print("C5 sBIT recovery (v16 >> 4 == 12-bit original), mismatches:", int(((back >> 4) != p12).sum()))
# ---- (d), (e) float16 and float32 TIFF with tifffile ----------------------------------
def tiff_roundtrip(a):
buf = io.BytesIO(); tifffile.imwrite(buf, a, compression="zlib"); buf.seek(0)
with tifffile.TiffFile(buf) as tf:
pg = tf.pages[0]
tags = (tf.byteorder, pg.bitspersample, int(pg.sampleformat), int(pg.compression))
return tags, pg.asarray()
tags16, t16 = tiff_roundtrip(x.astype(np.float16))
tags32, t32 = tiff_roundtrip(x)
print("C7 TIFF tags (byte order, BitsPerSample, SampleFormat, Compression): float16", tags16, "| float32", tags32)
# ---- (a), (a2) the ramp in 8-bit and 16-bit integer PNG -------------------------------
u8 = np.round(np.clip(x.astype(np.float64), 0, 1) * 255).astype(np.uint8)
u16 = np.round(np.clip(x.astype(np.float64), 0, 1) * 65535).astype(np.uint16)
mode_a, r8 = png_roundtrip(u8)
mode_a2, r16 = png_roundtrip(u16)
def mism(a, b):
return int((a != b).sum()), float(np.abs(a.astype(np.float64) - b.astype(np.float64)).max())
print("C8 container exact for the sample types written (mismatches, max |diff|):",
f"(a) {mode_a} {r8.dtype} {mism(r8, u8)}", f"| (a2) {mode_a2} {r16.dtype} {mism(r16, u16)}",
f"| (d) {t16.dtype} {mism(t16, x.astype(np.float16))}")
print("H1 exact round trips (mismatches, max |diff|):", f"(b) {mode_b} {grey16.dtype} {mism(grey16, p16[..., 0])}",
f"| (c) parsed by hand {back.dtype} {mism(back, p16)}", f"| (e) {t32.dtype} {mism(t32, x)}")
# ---- H3: what Pillow 12.0.0 makes of the 48-bit PNG -----------------------------------
im = Image.open(io.BytesIO(png48))
dec = np.asarray(im)
lin8 = np.floor(p16.astype(np.float64) * 255 / 65535 + 0.5)
print("H3 Pillow opens the 48-bit PNG as mode", im.mode, dec.dtype, dec.shape,
"| samples != v16 >> 8:", int((dec != (p16 >> 8)).sum()),
"| samples != linear rescale (13.12):", int((dec != lin8).sum()), "of", dec.size)
# ---- per-stop relative error of the ramp ----------------------------------------------
x64 = x.astype(np.float64)
stored = {"8-bit PNG": r8 / 255, "16-bit PNG": r16 / 65535, "float16 TIFF": t16, "float32 TIFF": t32}
E = {}
for name, v in stored.items():
rel = np.abs(v.astype(np.float64) - x64) / x64
E[name] = np.array([rel[:, stop == s].max() for s in range(-20, 10)])
print("max relative error per stop [2^s, 2^(s+1))")
print(" stop " + "".join(f"{n:>14s}" for n in stored))
for i, s in enumerate(range(-20, 10)):
print(f"{s:5d} " + "".join(f"{E[n][i]:14.4e}" for n in stored))
S = np.arange(-20, 10)
f16 = E["float16 TIFF"] / 2.0 ** -11
print(f"H2 float16 / 2^-11: normal stops -14..9 max {f16[S >= -14].max():.5f} (<= 1: {bool(np.all(f16[S >= -14] <= 1))});",
f"subnormal stops -20..-15 min {f16[S <= -15].min():.5f} (> 1: {bool(np.all(f16[S <= -15] > 1))})")
xmax = np.array([x64[0, stop == s].max() for s in S])
for name, half, lo_stops, zero_stops in (("8-bit PNG", 1 / 510, (-8, -1), (-20, -9)),
("16-bit PNG", 1 / 131070, (-16, -1), (-20, -18))):
e = E[name]
zs = (S >= zero_stops[0]) & (S <= zero_stops[1])
ls = (S >= lo_stops[0]) & (S <= lo_stops[1])
ratio = (e[ls] / (half / 2.0 ** S[ls])).max()
clip = np.abs(e[S >= 0] - (1 - 1 / xmax[S >= 0])).max()
print(f"{'H4' if name.startswith('8') else 'H5'} {name}: stops {zero_stops[0]}..{zero_stops[1]} min error {e[zs].min():.5f} (all 1: {bool(np.all(e[zs] == 1))});",
f"stops {lo_stops[0]}..{lo_stops[1]} max error / half-step bound {ratio:.8f} (<= 1 + 1e-12: {bool(ratio <= 1 + 1e-12)});",
f"stops 0..9 max |error - (1 - 1/x_max)| {clip:.1e} (<= 1e-12: {bool(clip <= 1e-12)})")
lo, hi = S <= -7, (S >= -5) & (S <= -1)
print("H5 float16 vs 16-bit PNG: float16 smaller in every stop <= -7:",
bool(np.all(E["float16 TIFF"][lo] < E["16-bit PNG"][lo])),
"| float16 larger in every stop -5..-1:", bool(np.all(E["float16 TIFF"][hi] > E["16-bit PNG"][hi])))
C1 data: ramp strictly increasing True | distinct 1920 | first == 2^-20 True | last < 2^10 True | samples per stop [64]
pattern: R codes = 0..4095 each once True | channel maxima [4095, 4095, 4095]
C2 CRC by hand vs zlib.crc32, mismatches: 0 of 4
first 33 bytes: 89 50 4e 47 0d 0a 1a 0a 00 00 00 0d 49 48 44 52 00 00 00 40 00 00 00 40 10 02 00 00 00 75 9b 3a ca
chunk IHDR length 13 CRC 759b3aca verified True
chunk sBIT length 3 CRC bfe9e621 verified True
chunk IDAT length 17369 CRC 783200e7 verified True
chunk IEND length 0 CRC ae426082 verified True
C3 CRC failures: 0
C6 IHDR: width 64 height 64 depth 16 colour type 2 compression 0 filter 0 interlace 0 | sBIT [12, 12, 12]
C3 filter bytes all 0: True
C4 left bit replication vs linear (12.4): max |diff| 1 | codes differing 952 of 4096
C5 sBIT recovery (v16 >> 4 == 12-bit original), mismatches: 0
C7 TIFF tags (byte order, BitsPerSample, SampleFormat, Compression): float16 ('<', 16, 3, 8) | float32 ('<', 32, 3, 8)
C8 container exact for the sample types written (mismatches, max |diff|): (a) L uint8 (0, 0.0) | (a2) I;16 uint16 (0, 0.0) | (d) float16 (0, 0.0)
H1 exact round trips (mismatches, max |diff|): (b) I;16 uint16 (0, 0.0) | (c) parsed by hand uint16 (0, 0.0) | (e) float32 (0, 0.0)
H3 Pillow opens the 48-bit PNG as mode RGB uint8 (64, 64, 3) | samples != v16 >> 8: 0 | samples != linear rescale (13.12): 1904 of 12288
max relative error per stop [2^s, 2^(s+1))
stop 8-bit PNG 16-bit PNG float16 TIFF float32 TIFF
-20 1.0000e+00 1.0000e+00 2.8533e-02 0.0000e+00
-19 1.0000e+00 1.0000e+00 1.3895e-02 0.0000e+00
-18 1.0000e+00 1.0000e+00 6.4934e-03 0.0000e+00
-17 1.0000e+00 1.0000e+00 3.0436e-03 0.0000e+00
-16 1.0000e+00 3.3016e-01 1.7182e-03 0.0000e+00
-15 1.0000e+00 1.9487e-01 9.0724e-04 0.0000e+00
-14 1.0000e+00 1.0963e-01 4.0398e-04 0.0000e+00
-13 1.0000e+00 5.4235e-02 4.0398e-04 0.0000e+00
-12 1.0000e+00 2.8548e-02 4.0398e-04 0.0000e+00
-11 1.0000e+00 1.3910e-02 4.0398e-04 0.0000e+00
-10 1.0000e+00 6.5088e-03 4.0398e-04 0.0000e+00
-9 1.0000e+00 3.0284e-03 4.0398e-04 0.0000e+00
-8 3.3043e-01 1.7029e-03 4.0398e-04 0.0000e+00
-7 1.9954e-01 8.9200e-04 4.0398e-04 0.0000e+00
-6 1.0883e-01 3.9834e-04 4.0398e-04 0.0000e+00
-5 5.8353e-02 2.1520e-04 4.0398e-04 0.0000e+00
-4 2.8173e-02 1.1125e-04 4.0398e-04 0.0000e+00
-3 1.3483e-02 5.1447e-05 4.0398e-04 0.0000e+00
-2 6.9064e-03 2.9374e-05 4.0398e-04 0.0000e+00
-1 3.9216e-03 1.5259e-05 4.0398e-04 0.0000e+00
0 4.9456e-01 4.9456e-01 4.0398e-04 0.0000e+00
1 7.4728e-01 7.4728e-01 4.0398e-04 0.0000e+00
2 8.7364e-01 8.7364e-01 4.0398e-04 0.0000e+00
3 9.3682e-01 9.3682e-01 4.0398e-04 0.0000e+00
4 9.6841e-01 9.6841e-01 4.0398e-04 0.0000e+00
5 9.8420e-01 9.8420e-01 4.0398e-04 0.0000e+00
6 9.9210e-01 9.9210e-01 4.0398e-04 0.0000e+00
7 9.9605e-01 9.9605e-01 4.0398e-04 0.0000e+00
8 9.9803e-01 9.9803e-01 4.0398e-04 0.0000e+00
9 9.9901e-01 9.9901e-01 4.0398e-04 0.0000e+00
H2 float16 / 2^-11: normal stops -14..9 max 0.82734 (<= 1: True); subnormal stops -20..-15 min 1.85804 (> 1: True)
H4 8-bit PNG: stops -20..-9 min error 1.00000 (all 1: True); stops -8..-1 max error / half-step bound 1.00000000 (<= 1 + 1e-12: True); stops 0..9 max |error - (1 - 1/x_max)| 1.1e-16 (<= 1e-12: True)
H5 16-bit PNG: stops -20..-18 min error 1.00000 (all 1: True); stops -16..-1 max error / half-step bound 1.00000000 (<= 1 + 1e-12: True); stops 0..9 max |error - (1 - 1/x_max)| 1.1e-16 (<= 1e-12: True)
H5 float16 vs 16-bit PNG: float16 smaller in every stop <= -7: True | float16 larger in every stop -5..-1: True
Checks. Every check held:
- C1: the ramp and pattern have the planned properties;
- C2: the hand CRC equals
zlib.crc32on all four chunks; - C3: every stored CRC verifies and every filter byte is 0;
- C4: left bit replication differs from the linear formula by at most one code, as §12.4 states, for 952 of the 4,096 codes;
- C5: shifting right by four recovers every 12-bit sample, as §13.12 says sBIT allows;
- C6: the parsed IHDR and sBIT are as written;
- C7: tifffile wrote BitsPerSample 16 and 32, SampleFormat 3 and Compression 8, little-endian;
- C8: the 8-bit, 16-bit and float16 files returned exactly the arrays written to them.
The first 33 bytes are the signature, IHDR’s length (13), the letters IHDR, width and height 64 (00 00 00 40), depth 16 (10), colour type 2 and three zero method bytes, and the CRC. Every loss in the per-stop table therefore happened before the file was written, in the conversion to the sample type, and none in the container.
H1 held. The 16-bit PNGs, written by Pillow and by hand, and the float32 TIFF returned every sample exactly. For the float32 ramp this is exact because the ground truth is itself float32.
H2 held. float16’s error divided by is at most 0.82734 in the normal stops and at least 1.85804 (stop ) in the subnormal stops, whose errors in the table grow to at stop . The table prints the same error, , in every normal stop because the samples repeat the same mantissas in every stop, so the relative error repeats too. Below the absolute step is fixed, and the relative error roughly doubles with each stop down, which is the “decreasing precision” of OpenEXR’s extra ten stops. Quantization states the same half-eps bound for the normal range and prints one image row’s error in float16; this example extends it per stop, through a TIFF file, and into the subnormal range. Mixed Precision Training shows what the same subnormal range does to gradients.
H3 held. Pillow 12.0.0 opened the 48-bit PNG as mode RGB with 8-bit samples, and every sample equalled the high byte of the stored 16-bit value. The low byte is discarded. PNG’s §13.12 calls this shift “slightly less accurate” than linear rescaling. The script prints 1,904 of the 12,288 samples where the two differ. The 16 bits written are there in the file, as the hand parser shows, but np.asarray(Image.open(...)) does not return them. 16-bit greyscale, by contrast, comes back as I;16 with every bit.
H4 held (its half-step and clipping parts are illustrations; see the process note). The 8-bit file stores every value below as zero, so the eleven darkest stops are lost entirely, and so are the lowest samples of the stop that begins at . Between and 1 the error stays within half a code, which is a third of the value in stop and 0.4% just below 1. Above 1 every value is clipped to 1, so a stop that is times brighter than white keeps 0.1% of its value. Gamma Correction shows how a nonlinear 8-bit encoding spreads its codes more evenly over the dark stops; in our reading, an encoding of still clips everything above white.
H5 held (its half-step and clipping parts are illustrations, as for H4). The 16-bit integer file reaches eight stops further down than the 8-bit one: stops to print an error of 1, and stop 0.33. Stop was not part of H5 as written; it prints 1 because is just below half a code. Above 1 it clips exactly as the 8-bit file does. Compared with float16 at the same 16 bits per sample, the integer file is more precise in stops to , where its fixed step is smaller than float16’s relative step; stop , which H5 left out as the crossover, also prints a slightly smaller error for the integer file. float16 is more precise in every stop up to , and it keeps the ten stops above 1 that any integer encoding normalized to white loses. In our reading this is the trade OpenEXR makes: a fixed number of steps per stop over a wide range, instead of fixed steps over .
Process note. The batch plan specified a ramp from to . That range lies entirely inside float16’s normal range, so the subnormal half of H2 could not have been tested on it. We corrected the plan before any script ran: the ramp now extends to , and its top 20 stops are the planned ramp. We also corrected the plan’s wording “zero below 1/255”, which holds for truncation; with rounding the limit is . Before the pilot we added four things to the plan’s example: the 16-bit integer ramp (a2), the hand-written CRC, left bit replication with the sBIT chunk, and the exactness checks C8. We also noted that Pillow cannot write 16-bit RGB PNG.
The hypotheses were written down at 08:31:51, before any script ran. One planning script then computed the same per-stop errors in memory, without files, and the plan was written from its output. Two tolerances were first written after that script: the half-step ratio of H4 and H5 is compared with and the clipping difference with . The script had printed a ratio of exactly 1 (round-half-to-even ties at , code 127.5) and a clipping difference of . Because that script printed these measures before they were fixed, the half-step and clipping parts of H4 and H5 are illustrations of the derivation, not tests of it.
The pilot was approved without changes to the set-up. The one change after the pilot was a print-format fix: C1’s count of samples per stop now prints as a plain list, [64], instead of a NumPy set. It changed only the format, not the measure.
Ecosystem and Integrations
- Pillow reads and writes PNG, TIFF, JPEG and many other formats, and returns images in modes rather than dtypes. In PngImagePlugin.py, 8-bit greyscale opens as
L, 16-bit greyscale asI;16(uint16), 16-bit RGB and RGBA as 8-bitRGBandRGBA(the example shows the RGB case), and 16-bit greyscale with alpha asRGBAthrough anLA;16Bunpacker. The writer savesI,I;16andI;16Bimages as 16-bit greyscale and has no 16-bit colour mode. Pillow’s handbook notes that Exif data can be read from PNG, and that reading or writing compressed TIFF files needs libtiff. Model Deployment coversImageOps.exif_transpose, and OpenCV covers how OpenCV’s decoders differ in channel order. - imageio wraps other libraries behind
imreadandimwrite. Its v3 Pillow plugin converts a PNG that Pillow opens in modeItoI;16before returning the array; Pillow 12.0.0 already opens 16-bit greyscale PNG asI;16. In our reading, a 48-bit PNG read through that plugin passes through the same Pillow decoder and returns 8-bit samples; this was not run. The plugin rotates by the Exif orientation and applies a PNG’s gAMA only when asked (rotate=Falseandapply_gamma=Falseare the defaults inread). For.exrfiles its first choice is the FreeImage plugin (then pyav and opencv), which needs the FreeImage library, downloaded separately if it is not on the system. - tifffile reads and writes TIFF, BigTIFF and their scientific variants. It maps SampleFormat and BitsPerSample to NumPy dtypes, (3, 16) to float16 and (3, 32) to float32 among them, writes tiles and pages, and uses the optional imagecodecs package for most codecs. Without it, zlib and LZMA compression still work through the pure-Python fallbacks in
_imagecodecs.py. - Dataset conventions sit on top of these containers. Depth Map and TUM RGB-D store depth as 16-bit PNG with a scale of 5,000 per metre, and the Disparity article shows KITTI’s scale of 256. Such scales are conventions of each dataset, not of PNG, so they must travel with the data.
Alternatives
For medical images, DICOM and NIfTI are described in Medical Image Preprocessing. For microscopy, OME-TIFF and OME-Zarr are described in Bioimage Analysis and Whole-Slide Imaging.
Versions and Compatibility
The example ran with NumPy 2.3.5, Pillow 12.0.0, imageio 2.37.2 and tifffile 2025.10.4. As of November 2026, Pillow 13.0.0, imageio 2.38.1 and tifffile 2026.11.12 have been released. Nothing here is claimed about them: the decode of 16-bit RGB PNG described above was read in Pillow 12.0.0’s source and should be checked again on any other version. The PNG Third Edition (2025) follows the Second Edition, a W3C Recommendation of 10 November 2003. The Library of Congress lists BigTIFF as an extension of TIFF 6.0, and Digital Camera describes DNG as another. The OpenEXR facts come from the documentation of v3.5.4, released on 12 November 2026.
Limitations
- A lossless container does not guarantee a lossless pipeline. The example’s files returned exactly what was written to them, but the 8-bit and 16-bit ramps had already lost their dark stops and everything above white, and Pillow returned 8 bits of a 16-bit RGB file. A round-trip test should compare the decoded array, dtype included, with the original.
- Integer containers need a scale. A 12-bit sensor value in a 16-bit PNG is either stored as written, which makes the image nearly black when displayed, or scaled up with sBIT recording the original depth. Depth, disparity and other measurements need their dataset’s scale factor. None of this is enforced by the format.
- Colour meaning is optional. Without gAMA, cHRM, sRGB, iCCP or cICP, a PNG’s RGB samples are device-dependent; TIFF needs its own colorimetry tags (Section 20). Whether values are linear or gamma-encoded has to be known from elsewhere.
- Metadata can be stale. PNG’s specification itself treats eXIf as historical unless its validity is known, and orientation handling differs between libraries (Model Deployment).
- TIFF’s flexibility is a portability cost. Float samples, tiles, unusual compressions and BigTIFF are all valid. Readers support different subsets, and a file that one library writes may not open in another.
Related
- Digital Camera
The camera as an imaging system, from lens, shutter and sensor to the image signal processor that demosaics, white-balances, colour-corrects, tone-maps and compresses its output; RAW against processed images, metadata, machine-vision interfaces, camera classes, and where processing breaks the assumptions of vision methods.
- High Dynamic Range Imaging
Capturing scenes whose brightness range exceeds what one exposure can record, by bracketing exposures, recovering the camera response and a radiance map, merging with noise-aware weights and removing ghosts, or with single-shot HDR sensors and learned single-image reconstruction, and compressing the result for display by tone mapping or exposure fusion.
- Gamma Correction
The nonlinear transfer functions that map light to stored code values and code values back to light, such as sRGB, BT.709 and BT.1886, and why they decide how many visible steps an 8-bit image has.
- Depth Map
An image whose pixels store the distance to the scene, usually the depth Z along the optical axis, so that each pixel with a known camera can be lifted to a 3D point.
- Whole-Slide Imaging
How slide scanners digitize glass microscope slides into gigapixel images, from tile and line scanning, stitching and focus to resolution in micrometers per pixel, pyramidal vendor, DICOM and OME-TIFF files, compression, color calibration and image quality, and how vision code reads them.
- Deploying Vision Models
Getting a trained vision model into use, from export and the choice of runtime to packaging the model with its preprocessing contract, serving it on a server or a device, keeping pre- and post-processing identical to training, and versioning, rolling back and monitoring it.
References
- Blume, C., Lemieux, P.-A., Lilley, C., Needham, C., Rosenthol, L., Seeger, C. A., Thompson, S. & Truta, C. (Eds.) (2025). Portable Network Graphics (PNG) Specification (Third Edition). W3C Recommendation, 24 June 2025.
- Aldus Corporation (1992). TIFF, Revision 6.0, Final, 3 June 1992. Copy hosted by the ITU, linked from the Library of Congress TIFF 6 format description.
- Library of Congress. BigTIFF (fdd000328). Sustainability of Digital Formats (last significant update 15 April 2024).
- Library of Congress. JPEG File Interchange Format (JFIF), Version 1.02 (fdd000018). Sustainability of Digital Formats (last significant update 7 May 2024).
- Library of Congress. JPEG file with Exif metadata (fdd000147). Sustainability of Digital Formats (last significant update 3 November 2023).
- Library of Congress. OpenEXR (fdd000583). Sustainability of Digital Formats (last significant update 12 May 2023).
- OpenEXR contributors. Technical Introduction to OpenEXR, website/TechnicalIntroduction.rst, OpenEXR v3.5.4 source.
- Pillow developers. Pillow 12.0.0 source code: src/PIL/PngImagePlugin.py and src/libImaging/Unpack.c.
- Pillow developers. Pillow 12.0.0 documentation: Image file formats, docs/handbook/image-file-formats.rst.
- Gohlke, C. tifffile 2025.10.4 source code: tifffile/tifffile.py and tifffile/_imagecodecs.py (read from the installed package).
- imageio contributors. imageio 2.37.2 source code: imageio/plugins/pillow.py, imageio/plugins/freeimage.py and imageio/config/extensions.py (read from the installed package).