2D and 3D Geometric Transformations

A geometric transformation moves every point of the plane or of space by one rule: rotate and shift it, scale it, shear it, or project it through a perspective. Computer vision uses a small number of these rules again and again, and they come in nested families. Euclidean (rigid) motions keep lengths; similarities also allow a change of scale and keep angles; affine maps also allow shear and keep parallel lines; projective maps keep straight lines but not parallelism. Each family has a fixed number of degrees of freedom, which sets how many correspondences pin it down, and each is a group, which is what lets transformations be chained, inverted and optimized. In 3D, the rigid motions $SE(3)$ describe camera poses, the rotations $SO(3)$ describe orientations, and the similarities $Sim(3)$ describe the extra scale freedom of a monocular camera.

Definition

A 2D geometric transformation is a map $T: \mathbb{R}^2 \to \mathbb{R}^2$ (or of the projective plane), and a 3D geometric transformation a map of $\mathbb{R}^3$ (or of projective space). The four classical families, from the most to the least constrained, are:

• Euclidean (rigid) transformations $\mathbf{x}' = R\mathbf{x} + \mathbf{t}$, a rotation $R$ and a translation $\mathbf{t}$;
• similarity transformations $\mathbf{x}' = sR\mathbf{x} + \mathbf{t}$, with a scale factor $s > 0$;
• affine transformations $\mathbf{x}' = A\mathbf{x} + \mathbf{t}$, with any invertible matrix $A$;
• projective transformations (homographies, collineations) $\tilde{\mathbf{x}}' \sim H\tilde{\mathbf{x}}$, with any invertible matrix $H$ acting on homogeneous coordinates up to scale.

Pure translations form a further family, more constrained than the Euclidean one. Szeliski (2022, §2.1.1–2.1.2) lists the five as a hierarchy for the plane and for space, and refers readers to Hartley and Zisserman (2004) for a more detailed description: each family contains the more constrained ones, and each is closed under composition and inversion, so each is a group and each smaller family is a subgroup of the larger.

Intuition

Picture a flat checkerboard and ask what a camera, or an algorithm, is allowed to do to its picture.

• Slide it and turn it on the table without lifting it: every square keeps its size and every corner stays a right angle. That is a Euclidean motion.
• Also move the camera closer or farther away, still looking straight down: squares grow or shrink together, but they stay squares. That is a similarity.
• Also look at the board from far away at a slant, through a long lens: squares become parallelograms, all the same shape, and parallel lines stay parallel. That is (very nearly) an affine map.
• Look at it at a slant from close up: the far squares shrink, lines that were parallel converge toward a vanishing point, and only straightness survives. That is a projective map.

Each step down the list adds freedom and gives up an invariant. The more freedom a family has, the more data it needs to be determined, and the more of the image’s true distortion it can absorb, including distortion caused by noise.

Formal Definition

Matrix forms

In homogeneous coordinates every family is a restricted matrix. For the plane, with $\mathbf{x} = (x, y)$ and $\tilde{\mathbf{x}} = (x, y, 1)^\top$,

$$\underbrace{\begin{bmatrix} R & \mathbf{t} \\ \mathbf{0}^\top & 1 \end{bmatrix}}_{\text{Euclidean}} \quad \underbrace{\begin{bmatrix} sR & \mathbf{t} \\ \mathbf{0}^\top & 1 \end{bmatrix}}_{\text{similarity}} \quad \underbrace{\begin{bmatrix} A & \mathbf{t} \\ \mathbf{0}^\top & 1 \end{bmatrix}}_{\text{affine}} \quad \underbrace{\begin{bmatrix} A & \mathbf{t} \\ \mathbf{v}^\top & w \end{bmatrix}}_{\text{projective}},$$

where $R \in SO(2)$ is a rotation, $A$ is any invertible $2 \times 2$ matrix and the projective matrix is invertible and defined only up to scale. The 3D forms are the same with $3 \times 3$ blocks and $4 \times 4$ matrices. The entries are written out for the plane in image warping, and Szeliski (2022, §2.1.1) writes the similarity’s linear part as $\begin{bmatrix} a & -b \\ b & a \end{bmatrix}$, a rotation and a scale combined into two free numbers with $a^2 + b^2$ no longer equal to 1. Only the last row distinguishes a projective map from an affine one: when $\mathbf{v} = \mathbf{0}$ the division by the third coordinate is by a constant, and the map is affine.

The hierarchy

Counting the free entries gives the degrees of freedom (DOF) of each family. The counts are those of Szeliski’s Tables 2.1 and 2.2; the minimal number of point correspondences follows from them, since each point correspondence gives two equations in 2D and three in 3D, and the invariants follow from the matrix forms (Szeliski’s tables list one invariant for each row; the others are worked out below).

Family	DOF in 2D	DOF in 3D	Minimal points, 2D / 3D	Preserves (and everything above it in the list)
Translation	2	3	1 / 1	orientation (the direction of every line)
Euclidean (rigid)	3	6	2 / 3	lengths, areas and volumes
Similarity	4	7	2 / 3	angles, ratios of lengths, shapes
Affine	6	12	3 / 4	parallelism of lines (and planes), ratios of lengths along a line or parallel lines, ratios of areas or volumes
Projective	8	15	4 / 5	straight lines (planes in 3D), incidence, the cross-ratio

The last column is read cumulatively: a property in a row is kept by that row’s family and by every family above it, so a similarity keeps angles, parallelism and straight lines, and only a Euclidean map also keeps lengths. The minimal sets need points in general position: no three collinear for a planar affine map or homography, no three collinear for a 3D rigid motion or similarity, and no four coplanar for a 3D affine or projective map. In 3D, a rigid motion needs three points although two would give six equations, because two points leave the rotation about the line through them free.

The invariants follow from the forms, which is our reasoning rather than a quotation:

• A Euclidean map moves the difference of two points by $R$, which keeps its length; it keeps handedness because $\det R = +1$.
• A similarity multiplies every length by the same $s$, so ratios of lengths and all angles survive, but lengths do not.
• An affine map sends the direction $\mathbf{d}$ of a line to $A\mathbf{d}$, independently of where the line is, so parallel lines stay parallel; it multiplies every area by $|\det A|$ (every volume in 3D), so ratios of areas survive, and it moves points along a line in proportion, so the midpoint of a segment maps to the midpoint of its image. Angles and the ratio of lengths in different directions are lost.
• A projective map keeps lines, because a line $\mathbf{l}$ maps to $H^{-\top}\mathbf{l}$, and keeps which points lie on which lines. Ratios of lengths along a line are lost, but the cross-ratio of four collinear points survives; projective geometry covers the cross-ratio, the line and plane at infinity and the circular points that tie these invariants to the hierarchy.

Group structure

A group is a set with an operation that is closed, associative, has an identity and has inverses. Barfoot (2024, §8.1.1, Table 8.1) checks these four axioms for rotations and rigid motions: a product of two rotation matrices is a rotation matrix, and a product of two rigid motions is a rigid motion,

$$\begin{bmatrix} R_1 & \mathbf{t}_1 \\ \mathbf{0}^\top & 1 \end{bmatrix} \begin{bmatrix} R_2 & \mathbf{t}_2 \\ \mathbf{0}^\top & 1 \end{bmatrix} = \begin{bmatrix} R_1 R_2 & R_1\mathbf{t}_2 + \mathbf{t}_1 \\ \mathbf{0}^\top & 1 \end{bmatrix}, \qquad \begin{bmatrix} R & \mathbf{t} \\ \mathbf{0}^\top & 1 \end{bmatrix}^{-1} = \begin{bmatrix} R^\top & -R^\top\mathbf{t} \\ \mathbf{0}^\top & 1 \end{bmatrix}.$$

The same block multiplication shows that products and inverses of similarities are similarities (the scales multiply), of affine maps are affine maps, and of invertible projective matrices are invertible projective matrices. Szeliski (2022, §2.1.1) states that the 2D families form a nested set of groups, each a subgroup of the next, and the same for 3D (§2.1.2). Three consequences matter in practice.

• Chains collapse. A sequence of warps or coordinate changes of one family is one matrix of that family, so a panorama’s chain of image-to-image homographies, or a robot’s chain of frame-to-frame poses, is a single transformation.
• Order matters. Apart from the translations and the planar rotations $SO(2)$, these groups are not commutative: $T_1 T_2 \neq T_2 T_1$ in general. Barfoot (2024, §8.1.1) notes that $SO(3)$ and $SE(3)$ are non-Abelian for this reason.
• Not every useful model fits the hierarchy. Szeliski (2022, §2.1.1) describes three other planar models. The stretch-and-squash map, separate scales along $x$ and $y$ plus a translation, is a restricted affine map, but he notes that it does not nest cleanly with the groups of the hierarchy; in our reading, that is because it neither contains the rotations nor lies inside the similarities. The eight-parameter planar surface flow, a small-motion approximation to a homography for a plane in small 3D motion, is linear in its parameters, which is its attraction. The bilinear interpolant, which can move four corner points anywhere, does not in general keep straight lines straight.

The group property is what Lucas–Kanade variants such as the inverse compositional algorithm rely on, as the image warping article describes: they compose an increment with the current warp and need the result to stay in the family.

Reflections. Allowing $\det R = -1$ adds reflections, which keep lengths but reverse handedness. The orthogonal matrices with either determinant form the orthogonal group $O(3)$; the rotations alone, with $\det R = +1$, form the special orthogonal group $SO(3)$ (Barfoot, 2024, §8.1.1, where a matrix with $\det = -1$ is called an improper rotation or rotary reflection). A least-squares fit of an orthogonal matrix to point data can return a reflection, so closed-form alignment methods such as the Kabsch algorithm guard against it: scikit-image’s implementation of Umeyama’s method, for example, flips the sign of the last singular direction when the cross-covariance matrix of the two point sets has a negative determinant, or, when that matrix has rank one less than the dimension, by the sign of $\det U \det V$, so that the result is a rotation (0.25.2 source).

Lie groups: SO(3), SE(3) and Sim(3)

The rotation, rigid and similarity groups are also smooth manifolds whose operations are smooth, which makes them Lie groups; a matrix Lie group is one whose elements are matrices, composed by multiplication and inverted by matrix inversion (Barfoot, 2024, §8.1.1). In 2D they are $SO(2)$ and $SE(2)$, the names under which, as Szeliski (2022, §2.1.1) notes, robotics tutorials treat the 2D rotations and rigid transforms. In 3D:

$$SO(3) = \{ R \in \mathbb{R}^{3 \times 3} : RR^\top = I,\ \det R = 1 \}, \qquad SE(3) = \left\{ \begin{bmatrix} R & \mathbf{t} \\ \mathbf{0}^\top & 1 \end{bmatrix} : R \in SO(3),\ \mathbf{t} \in \mathbb{R}^3 \right\},$$ $$Sim(3) = \left\{ \begin{bmatrix} sR & \mathbf{t} \\ \mathbf{0}^\top & 1 \end{bmatrix} : R \in SO(3),\ \mathbf{t} \in \mathbb{R}^3,\ s > 0 \right\}.$$

The first two follow Barfoot (2024, eqs. 8.1 and 8.5); the third is the definition of Strasdat, Montiel and Davison (2010, eq. 21). None of these sets is a vector space: the sum of two rotation matrices is not a rotation, and the zero matrix is not one (Barfoot, 2024, §8.1.1). Adding a correction to the entries of a rotation or pose matrix therefore leaves the group.

Each Lie group has a Lie algebra, a vector space that is the tangent space of the group at the identity (Barfoot, 2024, §8.1.2); its dimension is the group’s number of degrees of freedom:

• $\mathfrak{so}(3)$: skew-symmetric matrices $\boldsymbol{\phi}^\wedge$, one for each $\boldsymbol{\phi} \in \mathbb{R}^3$;
• $\mathfrak{se}(3)$: $4 \times 4$ matrices $\begin{bmatrix} \boldsymbol{\phi}^\wedge & \boldsymbol{\rho} \\ \mathbf{0}^\top & 0 \end{bmatrix}$ for $\boldsymbol{\xi} = (\boldsymbol{\rho}, \boldsymbol{\phi}) \in \mathbb{R}^6$;
• $\mathfrak{sim}(3)$: the 7-vector $(\boldsymbol{\omega}, \sigma, \boldsymbol{\upsilon})$ of Strasdat et al. (2010), with the scale written as $s = e^{\sigma}$ so that $\sigma$ ranges over all real numbers while $s$ stays positive.

The exponential map sends an algebra element to a group element, and the logarithm goes back. For rotations the exponential is the Rodrigues formula, covered with the logarithm and its numerical pitfalls in rotation representations. For rigid motions the rotation block is the same, but the translation is not $\boldsymbol{\rho}$ itself: Barfoot (2024, eqs. 8.33–8.34) shows that $\exp(\boldsymbol{\xi}^\wedge)$ has translation $J\boldsymbol{\rho}$, with $J = \sum_{n \ge 0} (\boldsymbol{\phi}^\wedge)^n / (n+1)!$, so the six numbers of $\boldsymbol{\xi}$ are not “a rotation vector and a translation” stacked together. Strasdat et al. (2010, eq. 22) give the closed form for $Sim(3)$: the linear block is $e^{\sigma}$ times the rotation of $\boldsymbol{\omega}$, and the translation is $W\boldsymbol{\upsilon}$ for a $3 \times 3$ matrix $W$ that depends on $\boldsymbol{\omega}$ and $\sigma$. The exponential map of $SO(3)$ and of $SE(3)$ is onto but not one-to-one, so every rotation or pose has many algebra representatives (Barfoot, 2024, §8.1.3), and Strasdat et al. state that the $Sim(3)$ exponential is onto, so a logarithm exists.

Estimators use these maps to keep each unknown inside its group while solving for an unconstrained increment: a nonlinear least-squares step solves for a 3-, 6- or 7-vector and applies it through the exponential to the current rotation, pose or similarity. Bundle adjustment and SLAM back ends work this way.

Properties

Degrees of freedom are gauge freedoms too

The degrees of freedom of a group also say what a set of measurements cannot determine. Strasdat et al. (2010, §I-B) call these gauge freedoms: transformations of the whole map, robot positions and landmarks together, that leave every measurement unchanged. A robot on a ground plane with a range-bearing sensor leaves 3 such freedoms (a planar rigid motion); a robot in 3D with calibrated stereo or a 3D laser scanner leaves 6 (a 3D rigid motion); a single camera leaves 7, a rigid motion plus the overall scale, that is, a 3D similarity. They argue that a map drifts in exactly these freedoms, so a monocular map drifts in scale as well as in position and orientation. This is why structure from motion and visual odometry from one camera recover the scene only up to a similarity, and why monocular trajectories are compared with ground truth after a $Sim(3)$ alignment, as camera pose describes.

Strasdat et al. used $Sim(3)$ to correct this drift at loop closures (2010, §IV-B). They converted each keyframe pose and each relative-pose constraint to a similarity with scale 1, gave the loop constraint the measured scale change, optimized the graph of 7-DOF constraints with residuals computed in $\mathfrak{sim}(3)$, and converted the corrected similarities back to rigid poses. On their Keble College sequence, a hand-held camera looping around an outdoor square, the loop closure detected a relative scale change of 1 : 0.37. They report that a 6-DOF pose-graph optimization closed the loop but left the scale drift, giving a deformed trajectory, while the 7-DOF optimization looked “significantly better” (2010, §V, Fig. 2); the comparison is qualitative, from the figure.

Choosing a family

Because the families are nested, it follows that a fit with more freedom can never fit the same correspondences worse by the same criterion, so a lower residual alone does not show that the extra freedom is needed. The usual rule, stated in image registration, is to use the least flexible family that explains the geometry. The geometry often decides it in advance (our summary of the cases in the matrix forms above and in the neighbouring articles):

• a rigid object, or a point cloud measured by a metric sensor, moves by a Euclidean transformation, as in iterative closest point;
• a monocular reconstruction relates to the world, or to another monocular reconstruction, by a similarity;
• a plane, or any scene seen by a camera that rotates about its centre, maps between images by a homography;
• a small, distant or shallow region maps nearly affinely, because the perspective division varies little across it;
• a general 3D scene seen from two positions is not related by any of these maps, because points at different depths move differently; epipolar geometry constrains such pairs instead.

Coordinates and conditioning

The same map has different matrices in different coordinate frames: if $C$ changes coordinates, $T$ becomes $CTC^{-1}$, which stays in the same family when $C$ is a similarity. Fitting methods are not always indifferent to the frame, however; the algebraic fit of a homography depends on the scaling of the coordinates, which is why it normalizes them first (homography, direct linear transform). The least-squares fits of the Euclidean and similarity families have closed forms, and the affine fit is linear least squares.

Examples

One set of correspondences, four families

The code renders a 9 × 7 checkerboard with 30 mm squares, seen by a 640 × 480 pinhole camera, once frontally from 0.9 m (view A) and four more times (view B). The four B views come from four camera motions chosen so that the exact image-to-image map is, in turn, a Euclidean map (roll about the optical axis and a sideways shift at the same distance), a similarity (the same with the camera moved back to 1.2 m), a nearly affine map (a 35° tilt seen from 4.5 m with a long lens) and a strongly projective one (the same tilt from 0.6 m). Each image gets Gaussian noise of standard deviation 0.01 on squares of intensity 0.2 and 0.8, and its 48 inner corners are detected with the Harris detector and refined to sub-pixel accuracy, so the correspondences are real measurements with a known ground truth. All four families are fitted from view A’s corners to view B’s by least squares on the same transfer error in view B: the Euclidean and similarity fits by Umeyama’s closed form as implemented in scikit-image 0.25.2, the affine fit by linear least squares, and the projective fit by Levenberg–Marquardt, run from both the normalized DLT and the affine fit, keeping the lower cost. For each fitted map, and for the detected corners themselves (the “data” row), the code measures what happens to the grid: the range of ratios of edge lengths, the largest change of a grid-corner angle, and how far the first and last rows (and columns) are from parallel.

import numpy as np
from scipy.optimize import least_squares
from skimage.feature import corner_harris, corner_peaks, corner_subpix
from skimage.transform import EuclideanTransform, SimilarityTransform, ProjectiveTransform

SQ, NX, NY = 0.03, 9, 7                        # 30 mm squares, 9 x 7 board
ii, jj = np.meshgrid(np.arange(1, NX), np.arange(1, NY))
corners = np.c_[(ii.ravel() - NX / 2) * SQ, (jj.ravel() - NY / 2) * SQ]   # 48 inner corners (m)


def rot(axis, deg):
    c, s = np.cos(np.radians(deg)), np.sin(np.radians(deg))
    if axis == "x":
        return np.array([[1, 0, 0], [0, c, -s], [0, s, c]])
    return np.array([[c, -s, 0], [s, c, 0], [0, 0, 1]])


def plane_to_image(f, R, t):
    """Homography K [r1 r2 t] from the board plane Z = 0 to pixels."""
    K = np.array([[f, 0, 320.0], [0, f, 240.0], [0, 0, 1]])
    return K @ np.c_[R[:, 0], R[:, 1], t]


def apply(H, p):
    q = np.c_[p, np.ones(len(p))] @ H.T
    return q[:, :2] / q[:, 2:]


def render(H, rng, ss=4):
    """640 x 480 image of the board, 4 x 4 supersampled, with sensor noise."""
    v, u = np.mgrid[0:480, 0:640].astype(float)
    img = np.zeros((480, 640))
    for dy in (np.arange(ss) + 0.5) / ss - 0.5:
        for dx in (np.arange(ss) + 0.5) / ss - 0.5:
            X = apply(np.linalg.inv(H), np.c_[u.ravel() + dx, v.ravel() + dy])
            i = np.floor(X[:, 0] / SQ + NX / 2).astype(int)
            j = np.floor(X[:, 1] / SQ + NY / 2).astype(int)
            on = (i >= 0) & (i < NX) & (j >= 0) & (j < NY)
            img += np.where(on, np.where((i + j) % 2 == 0, 0.2, 0.8), 0.5).reshape(480, 640)
    return img / ss**2 + 0.01 * rng.standard_normal((480, 640))


def detect(img, near):
    """Harris corners refined to sub-pixel accuracy, matched to the nearest expected corner."""
    peaks = corner_peaks(corner_harris(img, sigma=1.5), min_distance=8, threshold_rel=0.05)
    found = corner_subpix(img, peaks, window_size=13)[:, ::-1]          # (row, col) -> (x, y)
    return np.array([found[np.argmin(np.linalg.norm(found - p, axis=1))] for p in near])


def fit(model, a, b):
    """Least-squares fit of b ~ T(a); every model minimizes the same transfer error in b."""
    if model in ("Euclidean", "similarity"):                # Umeyama's closed form
        T = EuclideanTransform() if model == "Euclidean" else SimilarityTransform()
        T.estimate(a, b)
        return T.params
    aff = np.r_[np.linalg.lstsq(np.c_[a, np.ones(len(a))], b, rcond=None)[0].T, [[0, 0, 1]]]
    if model == "affine":
        return aff
    dlt = ProjectiveTransform()
    dlt.estimate(a, b)
    cost = lambda h: (apply(np.r_[h, 1].reshape(3, 3), a) - b).ravel()
    runs = [least_squares(cost, (H0 / H0[2, 2]).ravel()[:8], method="lm") for H0 in (dlt.params, aff)]
    return np.r_[min(runs, key=lambda r: r.cost).x, 1].reshape(3, 3)


def properties(pa, pb):
    """What the map pa -> pb did to the grid: edge length ratios, corner angles, parallel lines."""
    ga, gb = pa.reshape(NY - 1, NX - 1, 2), pb.reshape(NY - 1, NX - 1, 2)
    edges = lambda g: np.r_[(g[:, 1:] - g[:, :-1]).reshape(-1, 2), (g[1:] - g[:-1]).reshape(-1, 2)]
    ratio = np.linalg.norm(edges(gb), axis=1) / np.linalg.norm(edges(ga), axis=1)

    def angles(g):
        e1, e2 = (g[:-1, 1:] - g[:-1, :-1]).reshape(-1, 2), (g[1:, :-1] - g[:-1, :-1]).reshape(-1, 2)
        cos = np.sum(e1 * e2, 1) / np.linalg.norm(e1, axis=1) / np.linalg.norm(e2, axis=1)
        return np.degrees(np.arccos(cos))

    def direction(p):                                      # direction of the line fitted to points p
        return np.linalg.svd(p - p.mean(0))[2][0]

    def non_parallel(g):
        pairs = [(g[0], g[-1]), (g[:, 0], g[:, -1])]       # first and last rows, first and last columns
        return max(np.degrees(np.arccos(min(1, abs(direction(p) @ direction(q))))) for p, q in pairs)

    return ratio.min(), ratio.max(), np.abs(angles(gb) - angles(ga)).max(), non_parallel(gb)


rng = np.random.default_rng(0)
roll = rot("z", 20)
H_a = plane_to_image(800, np.eye(3), np.array([0, 0, 0.9]))        # view A: frontal, 0.9 m
views_b = {
    "rigid (roll, shift)": plane_to_image(800, roll, np.array([0.04, -0.02, 0.9])),
    "zoom (roll, 1.2 m)": plane_to_image(800, roll, np.array([0.04, -0.02, 1.2])),
    "far tilt (35 deg, 4.5 m, f 4000)": plane_to_image(4000, roll @ rot("x", 35), np.array([0, 0, 4.5])),
    "near tilt (35 deg, 0.6 m)": plane_to_image(800, roll @ rot("x", 35), np.array([0, 0, 0.6])),
}

pa = detect(render(H_a, rng), apply(H_a, corners))
print("view A corner error vs truth: RMS %.3f px" % np.sqrt(np.mean(np.sum((pa - apply(H_a, corners)) ** 2, 1))))
for name, H_b in views_b.items():
    pb = detect(render(H_b, rng), apply(H_b, corners))
    print(f"\n{name}: 48 corners, B error vs truth RMS "
          f"{np.sqrt(np.mean(np.sum((pb - apply(H_b, corners)) ** 2, 1))):.3f} px")
    print("  model        RMS resid   length ratio    max angle change   max non-parallel")
    lo, hi, ang, par = properties(pa, pb)
    print(f"  data (B)         -       {lo:.3f}-{hi:.3f}     {ang:6.2f} deg         {par:5.2f} deg")
    for model in ["Euclidean", "similarity", "affine", "projective"]:
        T = fit(model, pa, pb)
        rms = np.sqrt(np.mean(np.sum((apply(T, pa) - pb) ** 2, 1)))
        lo, hi, ang, par = properties(pa, apply(T, pa))
        print(f"  {model:11s} {rms:7.3f} px   {lo:.3f}-{hi:.3f}     {ang:6.2f} deg         {par:5.2f} deg")

Output:

view A corner error vs truth: RMS 0.040 px

rigid (roll, shift): 48 corners, B error vs truth RMS 0.033 px
  model        RMS resid   length ratio    max angle change   max non-parallel
  data (B)         -       0.997-1.006       0.40 deg          0.01 deg
  Euclidean     0.048 px   1.000-1.000       0.00 deg          0.01 deg
  similarity    0.048 px   1.000-1.000       0.00 deg          0.01 deg
  affine        0.048 px   1.000-1.000       0.00 deg          0.01 deg
  projective    0.047 px   1.000-1.000       0.01 deg          0.02 deg

zoom (roll, 1.2 m): 48 corners, B error vs truth RMS 0.027 px
  model        RMS resid   length ratio    max angle change   max non-parallel
  data (B)         -       0.747-0.754       0.38 deg          0.03 deg
  Euclidean    19.048 px   1.000-1.000       0.00 deg          0.01 deg
  similarity    0.039 px   0.750-0.750       0.00 deg          0.01 deg
  affine        0.039 px   0.750-0.750       0.01 deg          0.01 deg
  projective    0.038 px   0.750-0.750       0.02 deg          0.02 deg

far tilt (35 deg, 4.5 m, f 4000): 48 corners, B error vs truth RMS 0.028 px
  model        RMS resid   length ratio    max angle change   max non-parallel
  data (B)         -       0.803-1.014       1.12 deg          1.86 deg
  Euclidean     8.256 px   1.000-1.000       0.00 deg          0.01 deg
  similarity    6.629 px   0.935-0.935       0.00 deg          0.01 deg
  affine        0.453 px   0.819-1.000       0.05 deg          0.01 deg
  projective    0.043 px   0.807-1.010       0.94 deg          1.87 deg

near tilt (35 deg, 0.6 m): 48 corners, B error vs truth RMS 0.032 px
  model        RMS resid   length ratio    max angle change   max non-parallel
  data (B)         -       1.097-1.623       7.11 deg         13.96 deg
  Euclidean    32.962 px   1.000-1.000       0.00 deg          0.01 deg
  similarity   11.101 px   1.407-1.407       0.00 deg          0.01 deg
  affine        5.089 px   1.234-1.504       0.05 deg          0.01 deg
  projective    0.062 px   1.099-1.616       6.99 deg         13.96 deg

The detector’s error against the true projections is about 0.03–0.04 px, so a residual near 0.05 px is the noise floor: once a family reaches it, more freedom has nothing left to explain. In the rigid view all four families sit at that floor, and the data’s edge length ratios stay within 0.997–1.006; the data’s angle changes of about 0.4° are the same detection noise seen on edges only 27 px long. In the zoom view the Euclidean fit cannot change scale and misses by 19 px, while the similarity fits to the floor with a ratio of 0.750, the ratio of the two distances (0.9/1.2).

The tilted views separate the upper rows of the hierarchy. Seen from far away, the board’s edges shrink by different amounts in different directions (ratios 0.803–1.014), which a similarity’s single scale cannot follow, and its rows and columns converge by 1.86°, which the affine fit cannot follow; the affine residual of 0.45 px is ten times the floor. From close up, the lines converge by 14°, the affine fit misses by 5.1 px, and only the projective fit reaches the floor, reproducing the data’s convergence and angle changes.

The fitted maps show the invariants of their families: every Euclidean fit keeps lengths exactly (ratio 1.000), every similarity keeps one ratio and changes no angle, and every affine fit keeps rows and columns parallel. The 0.01° of non-parallelism in those rows is already present in view A’s detected corners; a parallelism-preserving map passes it on. The affine fits change the grid’s angles by only 0.05° in both tilted views because the tilt axis is parallel to the board’s rows: in our reading of the construction, the foreshortening stretches one grid direction relative to the other, which keeps right angles between them, whereas a tilt about a diagonal of the board would shear the grid and change them. The example runs in a few seconds with NumPy 2.3.5, SciPy 1.16.3 and scikit-image 0.25.2, and gives identical output on repeated runs.

Sim(3): exponential, closure and gauge

The second script implements the closed-form $Sim(3)$ exponential of Strasdat et al. (2010, eq. 22) and checks it against the matrix exponential of the $4 \times 4$ generator $\begin{bmatrix} \boldsymbol{\omega}^\wedge + \sigma I & \boldsymbol{\upsilon} \\ \mathbf{0}^\top & 0 \end{bmatrix}$. It then checks closure and the inverse, and moves a synthetic monocular reconstruction, 100 points seen by three cameras, by a random similarity to show the gauge freedom.

import numpy as np
from scipy.linalg import expm

rng = np.random.default_rng(0)


def hat(w):
    return np.array([[0, -w[2], w[1]], [w[2], 0, -w[0]], [-w[1], w[0], 0]])


def generator(w, sigma, u):
    """sim(3) element as a 4 x 4 matrix: rotation w, log-scale sigma, translation part u."""
    G = np.zeros((4, 4))
    G[:3, :3] = hat(w) + sigma * np.eye(3)
    G[:3, 3] = u
    return G


def rodrigues(w):
    th, A = np.linalg.norm(w), hat(w)
    return np.eye(3) + np.sin(th) / th * A + (1 - np.cos(th)) / th**2 * A @ A


def exp_sim3(w, sigma, u):
    """Closed form of Strasdat, Montiel and Davison (2010), eq. (22)."""
    th, A = np.linalg.norm(w), hat(w)
    a, b, c = np.exp(sigma) * np.sin(th), np.exp(sigma) * np.cos(th), (np.exp(sigma) - 1) / sigma
    W = ((a * sigma + (1 - b) * th) / (th * (sigma**2 + th**2)) * A
         + (c - ((b - 1) * sigma + a * th) / (sigma**2 + th**2)) * A @ A / th**2 + c * np.eye(3))
    S = np.eye(4)
    S[:3, :3], S[:3, 3] = np.exp(sigma) * rodrigues(w), W @ u
    return S


def split(S):
    """Scale, rotation and translation of a Sim(3) matrix [sR t; 0 1]."""
    s = np.cbrt(np.linalg.det(S[:3, :3]))
    return s, S[:3, :3] / s, S[:3, 3]


def sample():
    axis = rng.standard_normal(3)
    return axis / np.linalg.norm(axis) * rng.uniform(0.01, 3.0), rng.uniform(-1, 1), rng.standard_normal(3)


# 1. The closed form against the matrix exponential of the 4 x 4 generator.
err = max(np.abs(exp_sim3(*x) - expm(generator(*x))).max() for x in (sample() for _ in range(1000)))
print("closed-form exp vs expm, 1000 random sim(3) vectors: max abs difference %.1e" % err)

# 2. Sigma -> 0 gives the SE(3) exponential: same rotation, translation J u.
w, _, u = sample()
print("sigma = 1e-8 vs se(3) expm: max abs difference %.1e"
      % np.abs(exp_sim3(w, 1e-8, u) - expm(generator(w, 0.0, u))).max())

# 3. Closure and inverse: products and inverses of similarities are similarities.
S1, S2 = exp_sim3(*sample()), exp_sim3(*sample())
s, R, _ = split(S1 @ S2)
print("scale of S1 S2 = %.6f, product of scales = %.6f, |R^T R - I| = %.1e, det R = %.6f"
      % (s, split(S1)[0] * split(S2)[0], np.abs(R.T @ R - np.eye(3)).max(), np.linalg.det(R)))
s1, R1, t1 = split(S1)
S1_inv = np.eye(4)
S1_inv[:3, :3], S1_inv[:3, 3] = R1.T / s1, -R1.T @ t1 / s1
print("closed-form inverse: |S1^-1 S1 - I| = %.1e" % np.abs(S1_inv @ S1 - np.eye(4)).max())

# 4. Gauge freedom of a monocular reconstruction: move the world by a similarity
#    X' = s Q X + q and every camera by R' = R Q^T, t' = s t - R Q^T q. All
#    projections stay the same, so images alone cannot fix these 7 degrees of freedom.
K = np.array([[500, 0, 320], [0, 500, 240], [0, 0, 1.0]])
X = rng.uniform(-1, 1, (100, 3)) + [0, 0, 6]
cams = [(rodrigues(0.2 * rng.standard_normal(3)), 0.3 * rng.standard_normal(3))
        for _ in range(3)]                          # three cameras, small rotations
proj = lambda R, t, P: (lambda Y: Y[:, :2] / Y[:, 2:])((P @ R.T + t) @ K.T)
sg, Q, q = split(exp_sim3(*sample()))
Xg = sg * X @ Q.T + q
d = max(np.abs(proj(R @ Q.T, sg * t - R @ Q.T @ q, Xg) - proj(R, t, X)).max() for R, t in cams)
print("gauge change (scale %.3f): max change in reprojections %.1e px" % (sg, d))

Output:

closed-form exp vs expm, 1000 random sim(3) vectors: max abs difference 2.6e-14
sigma = 1e-8 vs se(3) expm: max abs difference 7.9e-09
scale of S1 S2 = 0.417766, product of scales = 0.417766, |R^T R - I| = 2.2e-16, det R = 1.000000
closed-form inverse: |S1^-1 S1 - I| = 2.2e-16
gauge change (scale 1.753): max change in reprojections 1.3e-13 px

The closed form agrees with the matrix exponential to rounding error over 1,000 random algebra vectors, rotation angles up to 3 rad and scales from $e^{-1}$ to $e$. With $\sigma = 10^{-8}$ it differs from the $\mathfrak{se}(3)$ exponential by about $10^{-8}$, which is the size of the scale change $e^{\sigma} - 1$ itself: as $\sigma \to 0$ the similarity exponential becomes the rigid one. The formula divides by $\sigma$ and by $\theta$, so in our reading an implementation also loses precision to cancellation when either is very small and would switch to series expansions there. The product of two similarities has the product of their scales and an exact rotation, and the block inverse undoes the similarity to rounding error. Moving the world by a similarity with scale 1.75, and every camera with it, changes the projections by about $10^{-13}$ px, rounding error: the seven numbers of a $Sim(3)$ are invisible to a single camera, which is the gauge freedom Strasdat et al. describe.

Common Misconceptions

• “An affine map preserves angles.” Only similarities and their subgroups do. An affine map keeps parallel lines and ratios along them, but generally changes angles and the ratio of lengths in different directions; a square becomes a parallelogram. The example’s affine fits to the tilted views keep the grid’s right angles only because the stretch happens to be along a grid direction.
• “The family with the lowest residual is the right one.” In nested families with the same criterion, the larger family always fits at least as well. The question is whether the residual drops below the measurement noise, and whether the scene geometry allows the family at all.
• “A rigid transformation is any length-preserving map.” Length-preserving maps include reflections, which a physical motion cannot produce. $SE(3)$ and $SO(3)$ require $\det R = +1$; fitting methods must enforce it.
• “The six numbers of a pose are a rotation vector and a translation.” In the $\mathfrak{se}(3)$ coordinates of Barfoot (2024), the translation of $\exp(\boldsymbol{\xi}^\wedge)$ is $J\boldsymbol{\rho}$, not $\boldsymbol{\rho}$; treating them as the same is approximately right only for small rotations.
• “Transformations can be applied in any order.” Rotating and then translating differs from translating and then rotating, and $T_{ab}T_{bc}$ is not $T_{bc}T_{ab}$. The order, and whether a matrix maps world to camera or camera to world, must be explicit, as camera pose discusses.
• “Monocular vision just needs a better estimator to get scale.” The scale is a gauge freedom of a single camera, not an estimation error: images alone do not determine it, and it drifts. It is fixed only by metric information from outside the images, such as the known baseline of a stereo rig, the anchor Strasdat et al. (2010) contrast with a single camera.

Where It Is Used

• Image alignment and stitching. Image registration chooses a family and fits it, feature matching with RANSAC estimates it from correspondences, and image warping resamples the image with it; panoramas chain homographies.
• Camera models and calibration. The extrinsic part of the pinhole camera model is an element of $SE(3)$; camera calibration estimates one per view of its target, and perspective-n-point estimates one from 2D–3D matches.
• Rotation estimation. Rotations in $SO(3)$ are parameterized and interpolated as rotation representations describes, and the essential matrix encodes a relative rotation and translation.
• Point-set alignment. The Kabsch algorithm and Umeyama’s method fit Euclidean and similarity transformations to corresponding 3D points in closed form, and iterative closest point alternates such fits with nearest-neighbour matching.
• SLAM and reconstruction. SLAM, visual odometry, bundle adjustment and structure from motion estimate poses in $SE(3)$ on the manifold; Strasdat et al. (2010) corrected the scale drift of monocular SLAM with a pose graph optimization over $Sim(3)$.
• Projective reconstruction. Uncalibrated two-view geometry, the fundamental matrix, recovers a scene only up to a 3D projective transformation; projective geometry describes how affine and metric structure are recovered from it.
• Learning. Geometric data augmentation applies rotations, shears and affine and projective warps to training images; the hierarchy says which image properties each kind keeps.