Local Binary Patterns

Local binary patterns (LBP) describe texture by the order of grey values around each pixel rather than by the values themselves. Each neighbour on a small circle is compared with the centre pixel and gives one bit: 1 if it is at least as bright, 0 otherwise. The bits, read around the circle, form a binary number, the pixel’s LBP code, and a region is described by the histogram of its codes. Because only signs of differences enter, any strictly increasing change of the grey scale leaves the code of a pixel with integer-position neighbours unchanged. Later work cites papers by Ojala, Pietikäinen and colleagues for the operator: Juefei-Xu, Boddeti and Savvides (2017) cite Ojala, Pietikäinen and Harwood (1996), and Cimpoi et al. (2014) and scikit-image’s documentation cite Ojala, Pietikäinen and Mäenpää (2002). Neither paper could be read for this article, so the operator is described here through scikit-image’s and OpenCV’s source code and through papers that define it. The codes are also used as a local feature descriptor: histograms over a grid of cells are used to describe faces (see face recognition), and a variant on block sums serves as the feature of boosted detection cascades.

Problem

Texture is a statistical property: a patch of gravel, fabric or skin is recognized by the kinds of small structures it contains and how often they occur, not by the exact grey value at each position. A texture descriptor therefore has to summarize many pixels, and it has to ignore the changes that do not alter the texture: brightness and contrast, the camera’s response curve, and, for many materials, the orientation of the patch. A plain grey-level histogram ignores position but responds to every change of illumination and none of the spatial structure. Filter-bank energies, such as those of a bank of Gabor filters, capture structure but scale with contrast. LBP takes a third route: it keeps the spatial structure at the scale of a few pixels and discards everything about grey values except their order.

Inputs and Outputs

• Input: a single-channel image (colour images are converted to grey first), and the operator’s parameters: the number of neighbours $P$, the radius $R$ in pixels and a mapping of codes to labels.
• Per-pixel output: an LBP image of the same size, with one integer label per pixel. With $P = 8$ the default code lies in $0 \ldots 255$.
• Descriptor: the normalized histogram of labels over a region, a whole texture sample or each cell of a grid, concatenated over cells and sometimes over several $(P, R)$ pairs. Histograms are compared with histogram distances, for example the $\chi^2$ distance of similarity and distance measures.

Intuition

Look at a pixel and its eight neighbours. If every neighbour is at least as bright, the pixel sits in a dark spot or on a flat area (code 255 with every bit set); if none is, it is a bright spot. If the bright neighbours form one contiguous arc, the pixel lies on an edge, a corner or a line end, and the arc’s length and position say which and in which direction. If bright and dark neighbours alternate several times around the circle, the neighbourhood is noise or very fine structure. A histogram of these labels over a region counts how often each kind of microstructure occurs, which is what distinguishes one texture from another, and it does so without a single absolute grey value.

Algorithm

1. Choose $P$ and $R$. Sample $P$ points evenly on a circle of radius $R$ around each pixel; points that do not fall on pixel centres get their value by bilinear interpolation.
2. For each pixel, compare each sampled neighbour $g_p$ with the centre $g_c$ and set bit $p$ to $1$ when $g_p \ge g_c$.
3. Combine the bits into the default code $\sum_p 2^p s_p$.
4. Optionally map the code to a coarser label: its rotation-invariant representative, its uniform label, or one label per uniform pattern plus one for the rest.
5. Count labels over each region of interest and normalize the counts.
6. Compare region histograms with a histogram distance, for example by nearest neighbour with $\chi^2$, or feed them to a classifier.

Mathematical Formulation

The operator

Let $g_c$ be the grey value of the centre pixel and $g_0, \ldots, g_{P-1}$ the values sampled on a circle of radius $R$. scikit-image places neighbour $p$ at the offset

$$(\Delta r_p, \Delta c_p) = \left(-R \sin\frac{2\pi p}{P},\; R \cos\frac{2\pi p}{P}\right)$$

in (row, column) coordinates, rounded to five decimals: $p = 0$ lies to the right of the centre and $p$ increases counter-clockwise, since rows point down. With $s(x) = 1$ for $x \ge 0$ and $0$ otherwise, the default code is

$$\mathrm{LBP}_{P,R} = \sum_{p=0}^{P-1} s(g_p - g_c)\, 2^p .$$

Juefei-Xu, Boddeti and Savvides (2017) define the same comparison on square patches: in a $3 \times 3$ patch the eight neighbours are compared with the centre in a fixed order, and in an $N \times N$ patch all $N^2 - 1$ other pixels are, giving a string of $N^2 - 1$ bits. The square $3 \times 3$ neighbourhood and the circle with $P = 8$, $R = 1$ share the four axis-aligned neighbours and differ at the diagonals: the square uses the corner pixels at distance $\sqrt 2$, the circle interpolates values at distance 1.

Grey-scale invariance

If $f$ is strictly increasing, $f(g_p) \ge f(g_c)$ exactly when $g_p \ge g_c$, so every bit computed from pixel values is unchanged by the grey-level map $g \mapsto f(g)$. Interpolated neighbours behave differently. A bilinear value $\sum_i w_i g_i$ is mapped to $\sum_i w_i g_i'$, which equals $f$ of the interpolated value only when $f$ is affine. Circular operators with interpolated neighbours are therefore exactly invariant to gain and offset, $g \mapsto a g + b$ with $a > 0$, but only approximately invariant to nonlinear monotonic maps such as a gamma curve. The Implementation section measures both cases. The invariance also says what the code cannot see: it ignores how large the differences are, so a faint pattern and a strong one with the same order relations get the same code. scikit-image’s var method supplies the missing contrast as the local variance

$$\mathrm{VAR}_{P,R} = \frac{1}{P} \sum_{p=0}^{P-1} (g_p - \mu)^2, \qquad \mu = \frac{1}{P} \sum_{p=0}^{P-1} g_p ,$$

which its docstring describes as rotation invariant but not grey-scale invariant and which its source computes with the biased ($1/P$) estimate, “following the LBP paper” in the words of a code comment.

Uniform patterns

The number of 0/1 transitions around the circle,

$$U = \sum_{p=0}^{P-1} \left| s(g_p - g_c) - s(g_{(p+1) \bmod P} - g_c) \right| ,$$

is even, because a circular string returns to its first bit. Patterns with $U \le 2$ are called uniform: all zeros, all ones, or one contiguous arc of ones. There are $2 + P(P-1)$ of them, since an arc of $k$ ones, $1 \le k \le P-1$, can start at any of $P$ positions. Cimpoi et al. (2014) credit the mapping of codes to uniform patterns to Ojala, Pietikäinen and Mäenpää (2002). scikit-image counts transitions only between neighbours $p$ and $p+1$ for $p < P-1$, omitting the pair that wraps around; by our reasoning this selects the same patterns, because adding the wrap-around pair can only raise a count of 1 to 2 or leave a count of 2 at 2, never push it past 2, since the circular count is even.

Mappings and their sizes

scikit-image’s method argument names four mappings of the default code (for $P = 8$, $R = 1$ the label counts, by our count, are in brackets):

• default: the code itself, $2^P$ labels [256]. Grey-scale invariant, not rotation invariant.
• ror: the smallest value among the $P$ circular bit rotations of the code, so that rotated versions of a pattern share one label. The number of labels is the number of distinct binary strings up to rotation [36 for $P = 8$].
• uniform (the riu2 mapping): the number of ones $\sum_p s_p$ if $U \le 2$, and $P + 1$ otherwise, giving $P + 2$ labels [10]. Rotating a uniform pattern does not change its number of ones, so the label is rotation invariant, and all non-uniform patterns share one label.
• nri_uniform: one label for each uniform pattern and one for all the others, $P(P-1) + 3$ labels [59], not rotation invariant. scikit-image cites Ahonen, Hadid and Pietikäinen (2004, 2006) for it.

Rotating a pattern by a multiple of $360°/P$ about the centre shifts its bits cyclically, so ror and uniform labels are designed to ignore such rotations. On a digital image the invariance is exact only when the rotation maps both the pixel grid and the sample positions onto themselves: a quarter turn, for $P$ a multiple of 4. Any other rotation of an image has to be resampled, the samples change value, and the invariance holds only approximately.

How often are patterns uniform?

For the $3 \times 3$ operator on independent, identically distributed noise with a continuous distribution, the share of uniform patterns can be computed exactly; the derivation is ours. The nine values are exchangeable and almost surely distinct, so the number $k$ of neighbours above the centre, which is 8 minus the centre’s rank among the nine, is uniform on $0, \ldots, 8$, and given $k$ the set of positions of the ones is a uniformly random $k$-subset of the eight positions. Exactly $8$ of the $\binom{8}{k}$ subsets form one arc for $1 \le k \le 7$, so

$$\Pr(\text{riu2} = k) = \frac{1}{9} \cdot \frac{8}{\binom{8}{k}} \;\; (1 \le k \le 7), \qquad \Pr(\text{riu2} = 0) = \Pr(\text{riu2} = 8) = \frac{1}{9},$$

and the share of uniform patterns is their sum, $0.55238$. Images with spatial structure can have far higher shares, as the example measures.

Histograms and distances

For a region $A$ the descriptor is $h_A(\ell) = |\{x \in A : \text{label}(x) = \ell\}| / |A|$. Ahonen and colleagues compared face histograms with a $\chi^2$ distance, optionally weighted per region, as the face recognition article describes. The example below uses the form with the factor $\tfrac12$, $\chi^2(h, k) = \tfrac12 \sum_\ell (h_\ell - k_\ell)^2 / (h_\ell + k_\ell)$, which the similarity and distance measures article discusses; the factor does not change nearest-neighbour decisions.

Parameters

Parameter	Effect	Choice
$P$ (neighbours)	Angular resolution; $2^P$ default codes and $P(P-1)+3$ nri-uniform labels	OpenCV’s LBPH defaults to 8 with $R = 1$; its header calls 8 sample points appropriate and warns that more points cost more computation
$R$ (radius)	Spatial scale of the microstructure	the example uses 1 and 2; histograms for several radii can be concatenated (multiresolution)
Mapping	Invariance against distinctiveness	uniform (riu2) when rotation should be ignored; nri_uniform or default when orientation carries information (scikit-image cites the face papers for nri_uniform)
Region or grid	What the histogram summarizes	whole texture sample for texture classification; a grid of cells for faces (OpenCV’s LBPH defaults to $8 \times 8$ cells, Ahonen et al.’s experiments used $7 \times 7$ regions, as the face recognition article reports)
Distance	How histograms are compared	$\chi^2$ (Ahonen et al.; OpenCV’s LBPH); SVMs with $\chi^2$-type kernels in Cimpoi et al. (2014)
Tie rule	What $g_p = g_c$ gives	$\ge$ in scikit-image and in the basic $3 \times 3$ function of OpenCV’s LBPH source; a floating-point tolerance in OpenCV’s circular operator (see Implementation)

Complexity

Computing the code takes $P$ samples, each a bilinear interpolation of four pixels, and $P$ comparisons per pixel, so $O(PN)$ time for an $N$-pixel image. A mapping is a table look-up from the default code; tables of size $2^P$ are practical for moderate $P$ (the example uses them for $P = 16$), while scikit-image computes every mapping from the bits for each pixel. A histogram costs $O(N)$ and has $2^P$, $P(P-1)+3$ or $P+2$ bins depending on the mapping, independent of image size; an $8 \times 8$ grid of cells with 59 nri-uniform labels gives $64 \times 59 = 3{,}776$ values. The multi-block variant compares sums of rectangles, each four look-ups in an integral image (see Haar-like features), so its cost per feature does not depend on block size.

Implementation

The example implements the operator in NumPy, with bilinear sampling written the way scikit-image’s _local_binary_pattern does it and the four mappings as look-up tables, and adds the square $3 \times 3$ operator whose neighbours all lie on pixel centres. It checks the code against skimage.feature.local_binary_pattern and then measures four properties: invariance to monotonic grey-level maps (part B), the share of uniform patterns against the closed form for noise and on images bundled with scikit-image (part C), the effect of a quarter turn (part D) and texture classification under rotation and a gamma curve (part E). Coordinates are (row, column) with rows pointing down; angles are measured counter-clockwise from the column axis.

One implementation detail had to be settled before the checks could be read. When a sampled value equals the centre, the bit is 1, but an interpolated value that equals the centre in exact arithmetic can come out a few units in the last place below or above it, and scikit-image’s compiled code and NumPy round differently. The code therefore treats differences smaller than $10^{-11}$ as ties (TIE), as OpenCV’s circular operator treats differences below single-precision epsilon, and prints results without (tol 0) and with the tolerance. It ran in about 15 seconds on one CPU thread with NumPy 2.3.5, SciPy 1.16.3 and scikit-image 0.25.2.

import os, sys
for v in ("OMP_NUM_THREADS", "OPENBLAS_NUM_THREADS", "MKL_NUM_THREADS", "VECLIB_MAXIMUM_THREADS"):
    os.environ[v] = "1"                               # one BLAS thread, before NumPy loads
import numpy as np
from math import comb
from scipy import ndimage
from skimage import data, color, util
from skimage.feature import local_binary_pattern

SEED = int(sys.argv[1]) if len(sys.argv) > 1 else 0   # base seed of every random image

# ---------- the operator ----------
def offsets(P, R, square=False):
    """Neighbour offsets (rows, cols), p = 0 to the east, counter-clockwise (scikit-image order)."""
    if square:                                        # 3x3 operator: integer positions only
        return np.array([0, -1, -1, -1, 0, 1, 1, 1.]), np.array([1, 1, 0, -1, -1, -1, 0, 1.])
    a = 2 * np.pi * np.arange(P) / P
    return np.round(-R * np.sin(a), 5), np.round(R * np.cos(a), 5)

TIE = 1e-11   # |g_p - g_c| below this is rounding error of the interpolation: a tie, bit 1

def diffs(img, P, R, square=False):
    """g_p - g_c for every neighbour p and pixel: bilinear g_p, zeros outside (as scikit-image)."""
    img = np.asarray(img, float)
    H, W = img.shape
    K = int(np.ceil(R)) + 1
    pad = np.pad(img, K)
    out = []
    for dr, dc in zip(*offsets(P, R, square)):
        r = np.arange(H)[:, None] + dr
        c = np.arange(W)[None, :] + dc
        r0, c0 = np.floor(r).astype(int) + K, np.floor(c).astype(int) + K
        r1, c1 = np.ceil(r).astype(int) + K, np.ceil(c).astype(int) + K
        fr, fc = r - np.floor(r), c - np.floor(c)
        top = (1 - fc) * pad[r0, c0] + fc * pad[r0, c1]
        bottom = (1 - fc) * pad[r1, c0] + fc * pad[r1, c1]
        out.append((1 - fr) * top + fr * bottom - img)
    return np.array(out)

def lbp_default(img, P, R, square=False, tol=TIE):
    """Default LBP code of every pixel: bit p = [g_p - g_c >= -tol]."""
    d = diffs(img, P, R, square)
    return ((d >= -tol).astype(np.int64) << np.arange(P)[:, None, None]).sum(0)

def mappings(P):
    """Look-up tables from the default code to the ror, riu2 and nri-uniform labels."""
    codes = np.arange(2 ** P)
    bits = (codes[:, None] >> np.arange(P)) & 1
    U = np.abs(bits - np.roll(bits, -1, axis=1)).sum(1)   # circular 0/1 transitions
    ones = bits.sum(1)
    rots = [(np.roll(bits, -k, axis=1) << np.arange(P)).sum(1) for k in range(P)]
    ror = np.min(rots, axis=0)
    riu2 = np.where(U <= 2, ones, P + 1)
    uni = np.flatnonzero(U <= 2)
    nri = np.full(2 ** P, len(uni))
    nri[uni] = np.arange(len(uni))
    return {"default": codes, "ror": ror, "uniform": riu2, "nri_uniform": nri}

LUT = {P: mappings(P) for P in (8, 16)}
NBINS = {"default": 256, "ror": 256, "uniform": 10, "nri_uniform": 59}  # for P = 8

def hist(labels, n):
    h = np.bincount(labels.ravel(), minlength=n).astype(float)
    return h / h.sum()

def chi2(h, k):
    s = h + k
    m = s > 0
    return 0.5 * np.sum((h[m] - k[m]) ** 2 / s[m])

def grey(name):
    im = getattr(data, name)()
    return util.img_as_ubyte(color.rgb2gray(im)) if im.ndim == 3 else im

M = 3                                                 # border dropped before any statistic
inner = lambda a: a[M:-M, M:-M]

# ---------- A. the NumPy operator against scikit-image ----------
print("A. pixels whose code differs from skimage.feature.local_binary_pattern (whole image)")
for name in ("camera", "coins"):
    img = grey(name)
    for P, R in ((8, 1), (16, 2)):
        d = diffs(img, P, R)
        row = []
        for tol in (0, TIE):
            code = lbp_default(img, P, R, tol=tol)
            row.append([int(np.sum(LUT[P][m][code] != local_binary_pattern(img, P, R, m)))
                        for m in ("default", "ror", "uniform")])
        ref = local_binary_pattern(img, P, R, "default").astype(np.int64)
        x = code ^ ref                                   # bits that differ (tolerance TIE)
        far = int(np.sum(((x[None] >> np.arange(P)[:, None, None]) & 1) & (np.abs(d) >= TIE)))
        gap = np.abs(d)[np.abs(d) >= TIE].min()          # smallest genuine difference
        same = code == ref
        mine, other = LUT[P]["nri_uniform"][code][same], local_binary_pattern(img, P, R, "nri_uniform")[same]
        pairs = len(np.unique(np.stack([mine, other]), axis=1).T)
        bij = pairs == len(np.unique(mine)) == len(np.unique(other))
        print(f"  {name:7s} P={P:2d} R={R}: default/ror/uniform tol 0 {row[0]}, tol {TIE:g} {row[1]}; "
              f"differing bits not ties {far}; smallest |g_p - g_c| above TIE {gap:.1e}; nri_uniform one-to-one: {bij}")

# ---------- B. monotonic grey-level changes ----------
cam = grey("camera").astype(float)
rng = np.random.default_rng(SEED)
lut = np.cumsum(rng.integers(1, 20, 256)).astype(float)   # random strictly increasing map
transforms = {"3g + 20": 3 * cam + 20, "255 (g/255)^0.5": 255 * (cam / 255) ** 0.5,
              "random increasing LUT": lut[cam.astype(int)]}
ops = {"3x3 square, P=8": (8, 1, True), "circle P=8, R=1": (8, 1, False), "circle P=16, R=2": (16, 2, False)}
print("\nB. share of interior pixels whose default code changes (camera)")
print(f"  {'':10s}{'operator':18s}" + "".join(f"{t:>24s}" for t in transforms))
for tol in (0, TIE):
    for op, (P, R, sq) in ops.items():
        c0 = inner(lbp_default(cam, P, R, sq, tol))
        print(f"  tol {tol:<6g}{op:18s}" + "".join(f"{np.mean(inner(lbp_default(t, P, R, sq, tol)) != c0):24.5f}"
                                                for t in transforms.values()))

# ---------- C. how common are uniform patterns? ----------
P = 8
p_k = np.array([1 / 9 * (1 if k in (0, 8) else 8 / comb(8, k)) for k in range(9)])
print(f"\nC. i.i.d. noise, 3x3 operator: closed form share of uniform patterns {p_k.sum():.5f}")
shares, hs = [], []
for s in range(5):
    noise = np.random.default_rng([SEED, 1, s]).random((512, 512))
    h = hist(LUT[8]["uniform"][inner(lbp_default(noise, 8, 1, True))], 10)
    hs.append(h); shares.append(1 - h[9])
print(f"  measured over 5 noise images: {min(shares):.5f} to {max(shares):.5f}; "
      f"largest riu2-bin error of the mean histogram {np.max(np.abs(np.mean(hs, 0)[:9] - p_k)):.5f}")
circ_noise = hist(LUT[8]["uniform"][inner(lbp_default(np.random.default_rng([SEED, 1, 0]).random((512, 512)), 8, 1))], 10)
print(f"  same noise, circle P=8, R=1 (interpolated diagonals): {1 - circ_noise[9]:.5f}")
print("  share of uniform patterns in bundled images:")
print(f"  {'image':9s}{'3x3':>8s}{'P=8,R=1':>10s}{'P=16,R=2':>10s}")
for name in ("camera", "coins", "moon", "page", "text", "astronaut", "chelsea", "coffee", "rocket"):
    img = grey(name).astype(float)
    s3 = 1 - hist(LUT[8]["uniform"][inner(lbp_default(img, 8, 1, True))], 10)[9]
    s8 = 1 - hist(LUT[8]["uniform"][inner(lbp_default(img, 8, 1))], 10)[9]
    s16 = 1 - hist(LUT[16]["uniform"][inner(lbp_default(img, 16, 2))], 18)[17]
    print(f"  {name:9s}{s3:8.3f}{s8:10.3f}{s16:10.3f}")

# ---------- D. a quarter turn of the image ----------
print("\nD. camera turned by 90 degrees: sum of |count differences| between histograms")
for tol in (0, TIE):
    for op, (P, R, sq) in ops.items():
        a = inner(lbp_default(cam, P, R, sq, tol))
        b = inner(lbp_default(np.rot90(cam), P, R, sq, tol))
        diff = []
        for m in ("default", "nri_uniform", "ror", "uniform"):
            n = 2 ** P
            diff.append(int(np.abs(np.bincount(LUT[P][m][a].ravel(), minlength=n)
                                   - np.bincount(LUT[P][m][b].ravel(), minlength=n)).sum()))
        print(f"  tol {tol:<6g}{op:18s} default {diff[0]:6d}  nri_uniform {diff[1]:6d}  ror {diff[2]:4d}  uniform {diff[3]:4d}")

Output:

A. pixels whose code differs from skimage.feature.local_binary_pattern (whole image)
  camera  P= 8 R=1: default/ror/uniform tol 0 [0, 0, 0], tol 1e-11 [0, 0, 0]; differing bits not ties 0; smallest |g_p - g_c| above TIE 6.0e-03; nri_uniform one-to-one: True
  camera  P=16 R=2: default/ror/uniform tol 0 [43, 43, 14], tol 1e-11 [1350, 1350, 391]; differing bits not ties 0; smallest |g_p - g_c| above TIE 1.0e-05; nri_uniform one-to-one: True
  coins   P= 8 R=1: default/ror/uniform tol 0 [0, 0, 0], tol 1e-11 [0, 0, 0]; differing bits not ties 0; smallest |g_p - g_c| above TIE 6.0e-03; nri_uniform one-to-one: True
  coins   P=16 R=2: default/ror/uniform tol 0 [9, 9, 4], tol 1e-11 [145, 145, 64]; differing bits not ties 0; smallest |g_p - g_c| above TIE 1.0e-05; nri_uniform one-to-one: True

B. share of interior pixels whose default code changes (camera)
            operator                           3g + 20         255 (g/255)^0.5   random increasing LUT
  tol 0     3x3 square, P=8                    0.00000                 0.00000                 0.00000
  tol 0     circle P=8, R=1                    0.00008                 0.07499                 0.08503
  tol 0     circle P=16, R=2                   0.00156                 0.06778                 0.17991
  tol 1e-11 3x3 square, P=8                    0.00000                 0.00000                 0.00000
  tol 1e-11 circle P=8, R=1                    0.00000                 0.04604                 0.08501
  tol 1e-11 circle P=16, R=2                   0.00000                 0.03711                 0.18032

C. i.i.d. noise, 3x3 operator: closed form share of uniform patterns 0.55238
  measured over 5 noise images: 0.55177 to 0.55304; largest riu2-bin error of the mean histogram 0.00036
  same noise, circle P=8, R=1 (interpolated diagonals): 0.74794
  share of uniform patterns in bundled images:
  image         3x3   P=8,R=1  P=16,R=2
  camera      0.735     0.828     0.604
  coins       0.730     0.834     0.594
  moon        0.954     0.989     0.769
  page        0.795     0.867     0.649
  text        0.770     0.862     0.642
  astronaut   0.800     0.879     0.740
  chelsea     0.840     0.906     0.775
  coffee      0.719     0.841     0.659
  rocket      0.825     0.880     0.659

D. camera turned by 90 degrees: sum of |count differences| between histograms
  tol 0     3x3 square, P=8    default  30636  nri_uniform  20926  ror    0  uniform    0
  tol 0     circle P=8, R=1    default  27426  nri_uniform  21372  ror    0  uniform    0
  tol 0     circle P=16, R=2   default  95268  nri_uniform  31860  ror 1574  uniform  296
  tol 1e-11 3x3 square, P=8    default  30636  nri_uniform  20926  ror    0  uniform    0
  tol 1e-11 circle P=8, R=1    default  27426  nri_uniform  21372  ror    0  uniform    0
  tol 1e-11 circle P=16, R=2   default  95390  nri_uniform  31912  ror    0  uniform    0

Agreement with scikit-image. For $P = 8$, $R = 1$ the NumPy codes equal scikit-image’s at every pixel of both images, under all three mappings that the check compares directly, and the nri-uniform labels differ only by a renaming. For $P = 16$, $R = 2$ a few dozen pixels of camera differ, and the tolerance raises the count to 1,350, but every differing bit is one where $|g_p - g_c|$ is below the tolerance: the two programs disagree only about exact ties, which they resolve by rounding error. The smallest genuine difference, $10^{-5}$ for $P = 16$, is six orders of magnitude above the tolerance, so the tolerance changes no real comparison; it only makes ties give 1, as the definition says, whatever the rounding.

Grey-level maps (B). The $3 \times 3$ operator, whose neighbours are pixel values, keeps every code under all three maps, as the formulation says. The circular operators keep every code under the affine map $3g + 20$ once ties are handled consistently; without the tolerance, a fraction of a percent of codes flip because the transformed interpolations round differently at ties. Under the square root and the random increasing table the interpolated diagonals do change bits: about 5% to 9% of codes change for $P = 8$, $R = 1$, and up to 18% for $P = 16$, $R = 2$, where most samples are interpolated. Exact invariance to every monotonic map belongs to neighbours on pixel centres; for interpolated operators it holds for gain and offset and only approximately for other monotonic maps.

Uniform patterns (C). On independent noise, five $512 \times 512$ images give shares between 0.5518 and 0.5530, against the closed-form 0.55238, and each riu2 bin of the mean histogram is within $4 \times 10^{-4}$ of its predicted probability. The circular operator on the same noise gives 0.748. In our reading, interpolated diagonals are averages of neighbouring pixels, so the eight samples are no longer independent and contiguous arcs become more likely. On the nine bundled images the share is 0.72–0.95 for the $3 \times 3$ operator and 0.83–0.99 for $P = 8$, $R = 1$, far above noise, and lower (0.59–0.78) for $P = 16$, $R = 2$, presumably because 16 samples have more ways to alternate. The smooth moon image is almost entirely uniform for the two operators of radius 1. On these images the single non-uniform bin of riu2 therefore holds only 1–17% of the pixels for $P = 8$, $R = 1$.

A quarter turn (D). Turning camera by 90° changes the default and nri-uniform histograms substantially and leaves the ror and uniform histograms identical for all three operators, because a quarter turn maps the pixel grid and the sampling set onto themselves and only rotates the bit string. For $P = 16$ the identity holds only with consistent tie handling: without it, interpolation in the turned image rounds ties differently and the ror histograms differ by 1,574 counts.

The second part of the script builds texture classes with known structure: six kinds of primitives (bars, crosses, L-shaped corners, spots, rings and pairs of parallel bars), 90 per $128 \times 128$ image at random positions, all oriented at one angle $\theta$, anti-aliased, with noise of standard deviation 0.02 and 8-bit quantization. Ten images per class at $\theta = 0$ form the training set. Test images (ten per class and condition, drawn afresh for each condition) are rendered at six angles, and in a seventh condition at $0°$ after the strictly increasing map $x \mapsto 0.1 + 0.8\sqrt{x}$. Each image is described by a 32-bin grey-level histogram (a baseline) and five LBP histograms, and classified by its $\chi^2$ nearest neighbour.

# ---------- E. textures of oriented primitives, rotated ----------
S = 128
CLASSES = ["bars", "crosses", "corners", "spots", "rings", "pairs"]

def segment_cover(r0, c0, r1, c1, width, rr, cc):
    d = np.array([r1 - r0, c1 - c0])
    t = np.clip(((rr - r0) * d[0] + (cc - c0) * d[1]) / (d @ d), 0, 1)
    dist = np.hypot(rr - r0 - t * d[0], cc - c0 - t * d[1])
    return np.clip(width / 2 - dist + 0.5, 0, 1)

def texture(cls, theta_deg, seed, n=90):
    """Primitives of one class at uniform random positions, oriented at theta
    (counter-clockwise from the column axis, rows pointing down); background 0.35,
    ink 0.75, Gaussian noise 0.02, quantized to 8 bits."""
    rng = np.random.default_rng(seed)
    rr, cc = np.mgrid[0:S, 0:S].astype(float)
    cover = np.zeros((S, S))
    th = np.deg2rad(theta_deg)
    u = np.array([-np.sin(th), np.cos(th)])          # along theta, in (row, col)
    v = np.array([-np.cos(th), -np.sin(th)])         # theta + 90 degrees
    for r, c in rng.uniform(-8, S + 8, (n, 2)):
        p = np.array([r, c])
        if cls == "bars":
            segs = [(p - 5 * u, p + 5 * u)]
        elif cls == "crosses":
            segs = [(p - 4 * u, p + 4 * u), (p - 4 * v, p + 4 * v)]
        elif cls == "corners":
            segs = [(p, p + 7 * u), (p, p + 7 * v)]
        elif cls == "pairs":
            segs = [(p - 5 * u - 2 * v, p + 5 * u - 2 * v), (p - 5 * u + 2 * v, p + 5 * u + 2 * v)]
        else:
            segs = []
        for a, b in segs:
            cover = np.maximum(cover, segment_cover(*a, *b, 2.0, rr, cc))
        if cls == "spots":
            cover = np.maximum(cover, np.clip(2.5 - np.hypot(rr - r, cc - c) + 0.5, 0, 1))
        if cls == "rings":
            cover = np.maximum(cover, np.clip(0.6 - np.abs(np.hypot(rr - r, cc - c) - 3.5) + 0.5, 0, 1))
    return np.clip(0.35 + 0.4 * cover + rng.normal(0, 0.02, (S, S)), 0, 1)

def to8(x):
    return np.round(255 * x)

def describe(img8):
    """Six descriptors of one 8-bit image (interior pixels only)."""
    c8, c16 = inner(lbp_default(img8, 8, 1)), inner(lbp_default(img8, 16, 2))
    return {"grey histogram (32)": hist((inner(img8) // 8).astype(int), 32),
            "LBP default (256)": hist(c8, 256),
            "LBP nri-uniform (59)": hist(LUT[8]["nri_uniform"][c8], 59),
            "LBP ror (36)": hist(LUT[8]["ror"][c8], 256),
            "LBP riu2 (10)": hist(LUT[8]["uniform"][c8], 10),
            "riu2 8,1 + 16,2 (28)": np.r_[hist(LUT[8]["uniform"][c8], 10),
                                           hist(LUT[16]["uniform"][c16], 18)] / 2}

ANGLES = [0, 15, 30, 45, 60, 90]
CONDS = [f"{a} deg" for a in ANGLES] + ["0 deg, gamma"]
train, ytr = [], []
for k, cls in enumerate(CLASSES):
    for i in range(10):
        train.append(describe(to8(texture(cls, 0, [SEED, 2, k, i])))); ytr.append(k)
ytr = np.array(ytr)

print("\nE. data check: ink fraction (mean per class, 0 deg) and bar orientation from the structure tensor")
ink = [np.mean([np.mean(texture(c, 0, [SEED, 9, i]) > 0.55) for i in range(3)]) for c in CLASSES]
print("  " + "  ".join(f"{c} {f:.2f}" for c, f in zip(CLASSES, ink)))
est = []
for a in ANGLES:
    t = texture("bars", a, [SEED, 8, a])
    gr = ndimage.gaussian_filter(t, 1.0, order=(1, 0))  # Gaussian derivatives, sigma 1
    gc = ndimage.gaussian_filter(t, 1.0, order=(0, 1))
    gx, gy = gc, -gr                                   # x to the right, y up
    J = np.array([[np.sum(gx * gx), np.sum(gx * gy)], [np.sum(gx * gy), np.sum(gy * gy)]])
    w, V = np.linalg.eigh(J)
    e = np.rad2deg(np.arctan2(V[1, 0], V[0, 0])) - a  # smallest eigenvector: along the bars
    est.append((e + 90) % 180 - 90)                   # signed error, wrapped to [-90, 90)
print("  bars, orientation error at " + ", ".join(f"{a}: {e:+.1f}" for a, e in zip(ANGLES, est)) + " degrees")

acc = {m: [] for m in train[0]}
for cond in CONDS:
    correct = {m: 0 for m in acc}
    for k, cls in enumerate(CLASSES):
        for i in range(10):
            a = 0 if "gamma" in cond else int(cond.split()[0])
            x = texture(cls, a, [SEED, 3, CONDS.index(cond), k, i])
            if "gamma" in cond:
                x = 0.1 + 0.8 * x ** 0.5                 # brighter, lower contrast, nonlinear
            d = describe(to8(x))
            for m in acc:
                nn = np.argmin([chi2(d[m], t[m]) for t in train])
                correct[m] += ytr[nn] == k
    for m in acc:
        acc[m].append(correct[m] / 60)

print("\nE. 1-NN accuracy with the chi-square distance; trained at 0 deg, 60 test images per column")
print(f"  {'descriptor':22s}" + "".join(f"{c:>8s}" if "gamma" not in c else f"{'0,gamma':>9s}" for c in CONDS))
for m, a in acc.items():
    print(f"  {m:22s}" + "".join(f"{x:8.2f}" for x in a[:-1]) + f"{a[-1]:9.2f}")

Output:

E. data check: ink fraction (mean per class, 0 deg) and bar orientation from the structure tensor
  bars 0.10  crosses 0.14  corners 0.13  spots 0.09  rings 0.11  pairs 0.19
  bars, orientation error at 0: -0.1, 15: -1.4, 30: -0.0, 45: +0.0, 60: +0.1, 90: -0.9 degrees

E. 1-NN accuracy with the chi-square distance; trained at 0 deg, 60 test images per column
  descriptor               0 deg  15 deg  30 deg  45 deg  60 deg  90 deg  0,gamma
  grey histogram (32)       0.88    0.90    0.83    0.87    0.85    0.85     0.17
  LBP default (256)         1.00    0.55    0.37    0.33    0.33    0.65     1.00
  LBP nri-uniform (59)      1.00    0.53    0.48    0.40    0.42    0.67     1.00
  LBP ror (36)              0.98    0.50    0.37    0.48    0.37    0.98     0.95
  LBP riu2 (10)             0.97    0.40    0.47    0.70    0.48    0.98     0.97
  riu2 8,1 + 16,2 (28)      1.00    0.98    0.85    0.87    0.75    1.00     1.00

The data check confirms that the textures have the planned properties: the bars come out at the requested angle to within 1.4°, and ink covers 9–19% of each image, so the background with its noise occupies most pixels.

Grey-level change. Every LBP descriptor classifies the gamma-mapped test images about as well as the unmapped test images at 0°, as planned (ror scores 0.95 against 0.98; the two columns use different test images, so the script does not show whether that difference comes from the map, although part B shows that interpolated codes are not exactly invariant to nonlinear maps), while the grey-level histogram falls from 0.88 to 0.17, close to chance for six classes.

Rotation. Trained at 0°, the default and nri-uniform histograms drop to 0.33–0.55 at 15–60°. At 90° they recover only to 0.65 and 0.67: by construction, crosses, spots and rings look the same after a quarter turn and bars, corners and pairs do not. The rotation-invariant mappings were expected to hold up at every angle. The plan’s hypothesis read “rotation-invariant mappings keep accuracy at every angle near their 0-deg value”, with a margin of 0.10. This check failed. ror and riu2 match their 0° accuracy at 90°, as part D predicts, but fall to 0.37–0.70 at 15°, 30°, 45° and 60°, at 15° and 30° no better than the rotation-variant histograms. The multiresolution riu2 histogram, which adds $P = 16$, $R = 2$, holds best among the LBP descriptors (0.75–0.98), yet it also falls more than 0.10 below its 0° accuracy at 30°, 45° and 60°. The grey-level histogram, which ignores where grey values occur, stays at 0.83–0.90. The output shows what the construction guarantees: exact invariance only for rotations that map the sampling set and the pixel grid onto themselves (part D, and the 90° column). In our reading, which the script does not test, the rest comes from resampling. The strokes of these primitives are about 2 pixels wide, the scale of the operator, and an oblique bar on a pixel grid has anti-aliased, staircase edges whose $3 \times 3$ patterns differ from those of an axis-aligned bar, so the riu2 histogram of one class changes with angle by more than the histograms of different classes differ at 0°. Training at one orientation is therefore not enough for grid-scale texture; larger radii, or training samples at several orientations, are possible remedies, and the example does not measure either.

The example was planned before it ran; a pilot with another seed (200) ran every part, the rotation hypothesis failed there as well and was reported to the editor, who kept the set-up unchanged, and the output above is the first run with seed 0, run twice with identical output. Before the pilot, the tie tolerance and the use of Gaussian derivatives in the orientation check were added after planning scripts showed rounding disagreements at ties and a bias in a central-difference orientation estimate; after the pilot, with the editor’s approval, the orientation check was changed to print signed errors instead of raw angles.

scikit-image

skimage.feature.local_binary_pattern(image, P, R, method) in version 0.25.2 implements the circular operator with the five methods default, ror, uniform, nri_uniform and var, read from its docstring and Cython source. Neighbour positions are rounded to five decimals and sampled by bilinear interpolation; samples that fall outside the image take the value 0, so border pixels compare against zeros. A bit is set when the sample minus the centre is $\ge 0$, with no tolerance, and the docstring warns that floating-point images “may give unexpected results” when small numerical differences between adjacent pixels are present, recommending integer images. The result is a float array of labels. For nri_uniform the source numbers uniform patterns by their number of ones and their rotation, puts the all-ones pattern at $P(P-1)+1$ and all non-uniform patterns at $P(P-1)+2$; for var a neighbourhood with zero variance gives NaN. skimage.feature.multiblock_lbp computes the multi-block code from an integral image (see Variants).

OpenCV

Two parts of OpenCV use the operator, read from the 4.14.0 sources:

• cv::face::LBPHFaceRecognizer (opencv_contrib, face module) computes circular LBP with defaults radius 1 and 8 neighbours. It interpolates bilinearly in single precision and sets a bit when the sample is greater than the centre or differs from it by less than single-precision epsilon; border pixels within the radius are dropped rather than padded. It applies no uniform mapping: each cell of a grid (default $8 \times 8$, with remainder pixels at the right and bottom ignored) gets a normalized histogram of all $2^P$ codes. Prediction is the nearest training histogram under HISTCMP_CHISQR_ALT, OpenCV’s alternative $\chi^2$, with an optional distance threshold above which the face is rejected.
• opencv_traincascade offers LBP features for boosted cascades alongside Haar-like and HOG features, as the Viola–Jones article notes. Each feature is a multi-block code on a $3 \times 3$ grid of equal rectangles: eight bits that compare each outer rectangle’s sum with the centre rectangle’s ($\ge$), computed from an integral image, and treated by the boosting code as a categorical feature with 256 values. The training tool enumerates every rectangle size and position that fits in the detection window.

Properties and Behavior

Monotonic invariance, exactly and approximately. Bits computed from pixel values are unchanged by any strictly increasing grey-level map; bits involving interpolated samples are unchanged by gain and offset only, as part B shows. This is the same invariance that makes the census transform useful for stereo and flow under exposure differences (see brightness constancy).

Rotation invariance, exactly and approximately. ror and riu2 labels are unchanged by rotations that map the pixel grid and the sample circle onto themselves (quarter turns, for $P$ a multiple of 4). Other rotations of a sampled image change the samples themselves, and for fine, grid-scale structure the change can be large (part E).

Ties and quantization. With $s(0) = 1$, a perfectly flat area gives the all-ones code, so flat regions fall into a uniform bin rather than into noise. Ties occur wherever neighbouring values are equal, as in flat areas of 8-bit images, and when samples are interpolated the result of a tie depends on rounding unless the implementation handles it, as OpenCV’s circular operator does.

Noise sensitivity in flat areas. In nearly flat regions, differences of a grey level or two decide every bit, so sensor noise turns a flat patch into an arbitrary pattern (in part C, independent noise is uniform only 55% of the time with the $3 \times 3$ operator). By our reasoning, this is where VAR or a contrast threshold helps, since LBP alone cannot tell weak from strong differences.

Locality. A code describes structure within radius $R$. Larger-scale texture needs larger radii, several radii combined, or blocks instead of pixels.

Limitations

• No contrast information. The code discards the size of differences; textures that differ mainly in contrast look alike unless VAR or another contrast measure is added.
• Fragile in flat, noisy regions, where noise decides the bits (see Properties and Behavior).
• Small spatial support. With $R = 1$ or 2 the operator sees only micro-structure; Cimpoi et al. (2014) found uniform LBP ($\mathrm{LBP}^u$, for which they used the rotation-invariant multiple-radii version of Matthews et al.) far weaker than dense SIFT on their Describable Textures Dataset: with their best kernel (exponential $\chi^2$), 24.8% mAP for $\mathrm{LBP}^u$ and 34.7% for LBP-VQ, uniform LBP computed in $8 \times 8$ neighbourhoods and vector-quantized into 512 visual words, against 53.8% for SIFT (Table 1, mean average precision of 47 attribute classifiers, averaged over ten splits, bag-of-visual-words encoding). They add that $\mathrm{LBP}^u$ was developed on the Outex data, on which it performed well.
• Hand-designed binarization and coding. Lu, Liong and Zhou (2015) name two shortcomings for face recognition: both the binarization and the encoding of the codes are hand-crafted, and the two are carried out as separate stages. Their method learns both from raw pixels.
• Interpolation and orientation. Circular operators are only approximately invariant to nonlinear grey maps and to rotations that do not map the samples onto themselves (part E).

Variants

Multiresolution histograms. Concatenating histograms from several $(P, R)$ pairs describes structure at several scales; in part E the combination of $P = 8$, $R = 1$ and $P = 16$, $R = 2$ was the most robust LBP descriptor under rotation, although it too failed the planned rotation check.

Spatial histograms for faces. Dividing an aligned face into a grid of cells, computing a histogram in each and concatenating them gives a descriptor that keeps coarse layout; the face recognition article describes Ahonen, Hadid and Pietikäinen’s (2006) version, and OpenCV’s LBPH implements one. LBP histograms also appear as baselines and components elsewhere on this site: the baselines of a facial expression dataset, RAF-DB, include LBP histograms on cells, DeepFace located its fiducial points with a regressor on LBP histograms and borrowed the weighted $\chi^2$ similarity used with them, and selective search tested LBP as its texture similarity.

High-dimensional LBP. Chen, Cao, Wen and Sun (2013) sampled patches at 27 facial landmarks and five image scales (normalized faces with sides of 300, 212, 150, 106 and 75 pixels), cut each $40 \times 40$ patch into $4 \times 4$ cells and concatenated per-cell descriptors into a feature of about 100,000 dimensions, which they reduced by PCA and classified with a Joint Bayesian model. On LFW’s unrestricted protocol, LBP reached 93.18% verification accuracy in this form against 88.33% for their lower-dimensional baseline sampled on a regular grid (Table 1), and 95.17% against 89.39% when trained on their WDRef dataset and tested on LFW (Table 3). The gain held for all five descriptors they compared (LE, LBP, SIFT, HOG and Gabor), so in our reading it comes from the sampling and the dimension rather than from LBP itself.

Multi-block LBP (MB-LBP). Replacing pixels by sums over a $3 \times 3$ grid of equal rectangles gives a code that describes structure at any rectangle size in constant time with an integral image. scikit-image’s multiblock_lbp, whose docstring cites Zhang et al. (2007), and OpenCV’s cascade features implement it.

Census transform. The census transform, used as a matching cost in cost volumes for stereo, records comparisons of neighbours with a centre pixel as bits (in that article’s definition a bit marks a darker neighbour), but keeps them per pixel and compares two pixels’ strings by the Hamming distance instead of pooling them into a histogram.

Learned binary features. Lu, Liong and Zhou (2015) learn binary codes for local face patches and the codebook that pools them jointly, from raw pixels. Juefei-Xu, Boddeti and Savvides (2017) carried the idea into convolutional networks: their local binary convolution layer applies fixed, sparse, binary filters, inspired by LBP’s comparisons, followed by a nonlinearity and learnable linear weights that combine the responses. They report 9× to 169× fewer learnable parameters than a standard convolutional layer with $3 \times 3$ to $13 \times 13$ filters, and performance parity with ordinary CNNs on MNIST, SVHN, CIFAR-10 and ImageNet in the abstract’s summary.