Camera Intrinsic and Extrinsic Parameters

A camera’s parameters say how a point in the world becomes a pixel. They split into two groups: extrinsic parameters, which say where the camera is and which way it points, and intrinsic parameters, which describe the camera itself, turning a direction in front of the camera into a pixel position. Together they form the $3 \times 4$ camera matrix $P = K[R \mid \mathbf{t}]$, which extends the pinhole camera model from a camera fixed at the origin with metric image coordinates to a real camera anywhere in the scene producing pixel coordinates.

Definition

The extrinsic parameters are a rotation $R$ and a translation $\mathbf{t}$ that transform a point from world coordinates to camera coordinates. The intrinsic parameters are the focal lengths in pixels $f_x$ and $f_y$, the principal point $(c_x, c_y)$, and the skew $s$, collected in the upper-triangular intrinsic matrix (or calibration matrix) $K$. Lens distortion coefficients are usually counted as further intrinsic parameters. Intrinsics stay fixed as the camera moves, as long as the focus and zoom do not change; extrinsics change with every movement.

Intuition

Imaging a point happens in three steps. First, express the point relative to the camera: where it lies to the camera’s right, below it, and in front of it. That is a rigid motion, the extrinsics. Second, project it through the pinhole: divide by depth to get its direction, a 2D point on a plane one unit in front of the camera. Third, convert that direction into a pixel: scale by the focal length in pixels and shift so that the origin is at the image corner instead of the optical axis. That is the job of the intrinsics. Moving the camera changes only the first step; swapping the lens or resizing the image changes only the third.

Formal Definition

The camera matrix

With a world point $\mathbf{X} = (X, Y, Z)^\top$ and its pixel $(u, v)$ in homogeneous coordinates,

$$\lambda \begin{bmatrix} u \\ v \\ 1 \end{bmatrix} = K \,[\,R \mid \mathbf{t}\,] \begin{bmatrix} \mathbf{X} \\ 1 \end{bmatrix}, \qquad K = \begin{bmatrix} f_x & s & c_x \\ 0 & f_y & c_y \\ 0 & 0 & 1 \end{bmatrix},$$

where $\lambda$ is the depth of the point in the camera frame. This is the form used by Hartley and Zisserman (2004) and, without skew, by OpenCV’s calib3d module.

Extrinsics

The extrinsics map world coordinates to camera coordinates:

$$\mathbf{X}_c = R\,\mathbf{X} + \mathbf{t}.$$

$R$ is a $3 \times 3$ rotation matrix ($R^\top R = I$, $\det R = 1$) and $\mathbf{t}$ is the position of the world origin expressed in camera coordinates. It is not the camera’s position. The camera center $\mathbf{C}$ is the world point that maps to $\mathbf{X}_c = \mathbf{0}$, so

$$\mathbf{C} = -R^\top \mathbf{t}, \qquad P = K R\,[\,I \mid -\mathbf{C}\,].$$

The rows of $R$ are the camera’s axes written in world coordinates; the third row is the direction of the optical axis. The homogeneous camera center $(\mathbf{C}, 1)$ is the null vector of $P$: $P\,(\mathbf{C}, 1)^\top = \mathbf{0}$, so the center is the one point whose image is undefined.

Intrinsics

In camera coordinates the projection to pixels is

$$u = f_x \frac{X_c}{Z_c} + s \frac{Y_c}{Z_c} + c_x, \qquad v = f_y \frac{Y_c}{Z_c} + c_y.$$

• Focal lengths in pixels. A lens of focal length $f$ (in millimeters) on a sensor with pixel pitch $p_x \times p_y$ (in millimeters per pixel) gives $f_x = f / p_x$ and $f_y = f / p_y$. Hartley and Zisserman write the same thing as $\alpha_x = f m_x$, with $m_x$ pixels per unit length. The ratio $f_y / f_x$ is the aspect ratio, 1 for square pixels.
• Principal point. $(c_x, c_y)$ is the pixel where the optical axis meets the image. It is usually near the image center but rarely exactly at it.
• Skew. $s$ is nonzero only if the pixel grid’s axes are not perpendicular. For modern sensors $s = 0$, and many libraries, OpenCV included, omit it.

Normalized image coordinates

Applying $K^{-1}$ to a pixel removes the intrinsics:

$$\hat{\mathbf{x}} = K^{-1} \begin{bmatrix} u \\ v \\ 1 \end{bmatrix} = \begin{bmatrix} X_c / Z_c \\ Y_c / Z_c \\ 1 \end{bmatrix}.$$

These normalized coordinates are what an ideal camera with $f = 1$ and the principal point at the origin would measure, and $\hat{\mathbf{x}}$ is the direction of the ray through the pixel in the camera frame. Methods that care only about rays, such as relative pose estimation with the essential matrix and triangulation, work in normalized coordinates once $K$ is known.

Distortion

Real lenses add lens distortion, which OpenCV models as a nonlinear function applied to the normalized coordinates $(X_c/Z_c, Y_c/Z_c)$ before $K$: radial coefficients $k_1, \dots, k_6$, tangential coefficients $p_1, p_2$, thin-prism coefficients $s_1, \dots, s_4$, and two sensor-tilt angles. Because these depend on the lens and not on the pose, they are intrinsic parameters, but they are not part of $K$, and $P$ alone no longer describes the camera once they are nonzero.

Properties

Degrees of freedom

$K$ has 5 parameters ($f_x, f_y, c_x, c_y, s$), $R$ has 3 (a rotation), and $\mathbf{t}$ has 3, for 11 in total. This matches $P$: it has 12 entries but is defined only up to scale, leaving 11 (Hartley and Zisserman, 2004). Conversely, any $3 \times 4$ matrix whose left $3 \times 3$ block is invertible can be written, up to scale, as $K[R \mid \mathbf{t}]$, as the decomposition below shows.

Field of view

For an image $W$ pixels wide and $H$ high, with the principal point at the center,

$$\theta_x = 2 \arctan\!\left( \frac{W}{2 f_x} \right), \qquad \theta_y = 2 \arctan\!\left( \frac{H}{2 f_y} \right).$$

The field of view depends only on the ratio of image size to focal length, so it is unchanged by resizing.

Resizing and cropping

$K$ belongs to a specific image resolution. Scaling an image by a factor $a$ multiplies $f_x$, $f_y$, $c_x$, and $c_y$ by $a$, as the OpenCV documentation notes; non-uniform scaling multiplies the $x$ and $y$ entries by their own factors. This is exact if pixel centers lie at half-integer coordinates, so that the image spans $[0, W]$. If instead pixel centers lie at integers, as in OpenCV, the image spans $[-\tfrac{1}{2}, W - \tfrac{1}{2}]$, and a resize that maps the image’s outer edges onto each other sends $x$ to $a\,(x + \tfrac{1}{2}) - \tfrac{1}{2}$; the principal point then becomes $c' = a\,(c + \tfrac{1}{2}) - \tfrac{1}{2}$, a sub-pixel difference that matters for precise work (see digital image for the two conventions). Cropping a window whose top-left corner is at $(x_0, y_0)$ leaves the focal lengths unchanged and shifts the principal point to $(c_x - x_0,\, c_y - y_0)$. Using a $K$ calibrated at one resolution on images of another is a common silent error.

Decomposing P

Given a camera matrix, for example one estimated from point correspondences, $K$, $R$, and $\mathbf{t}$ can be recovered. Write $P = [\,M \mid \mathbf{p}_4\,]$ with $M = KR$. An RQ decomposition factors $M$ into an upper-triangular matrix times an orthogonal matrix, which is exactly the form $KR$ (Hartley and Zisserman, 2004). The factorization is unique only up to the signs of the columns of $K$ and rows of $R$, so the signs are fixed by requiring a positive diagonal in $K$ and $\det R = +1$, flipping the overall sign of $P$ if necessary, since $P$ and $-P$ represent the same camera. Then $\mathbf{t} = K^{-1}\mathbf{p}_4$, and dividing $K$ by its bottom-right entry restores the usual normalization. OpenCV provides decomposeProjectionMatrix, built on its RQDecomp3x3.

Conventions

The world-to-camera transform $[R \mid \mathbf{t}]$ used above is the convention of Hartley and Zisserman and of OpenCV, whose documentation describes $R$ and $\mathbf{t}$ as the change of coordinates from world to camera. Many robotics and graphics tools instead store the camera pose, the camera-to-world transform

$$\mathbf{X} = R_{wc}\,\mathbf{X}_c + \mathbf{t}_{wc}, \qquad R_{wc} = R^\top, \quad \mathbf{t}_{wc} = \mathbf{C} = -R^\top \mathbf{t},$$

whose translation is the camera’s position. Both are $4 \times 4$ rigid transforms and each is the inverse of the other. Confusing them produces cameras that sit in the wrong place and point the wrong way, often without any error message. Axis orientation is a separate convention: OpenCV’s camera frame has $Y$ down and $Z$ forward, while OpenGL’s camera looks along $-Z$ with $Y$ up (see pinhole camera model). When combining data from different sources, check which transform direction and which axes each one uses.

Examples

The NumPy code below builds $K$ from a lens and sensor, sets up world-to-camera extrinsics, projects points, recovers the camera center, computes the field of view, shows how $K$ changes when the image is halved and cropped, and decomposes $P$ back into $K$, $R$, and $\mathbf{t}$.

import numpy as np
from scipy.linalg import rq
from scipy.spatial.transform import Rotation

np.set_printoptions(precision=3, suppress=True)

# Intrinsics: 4 mm lens, 2 um pixels, 1280 x 960 image
f_mm, pitch_mm = 4.0, 0.002
fx = fy = f_mm / pitch_mm                       # 2000 px
K = np.array([[fx, 0.0, 640.0],
              [0.0, fy, 480.0],
              [0.0, 0.0, 1.0]])

# Extrinsics, world-to-camera: X_cam = R X_world + t
R = Rotation.from_euler("xyz", [5, -5, 2], degrees=True).as_matrix()
C_true = np.array([1.0, -0.5, -6.0])            # camera centre in the world
t = -R @ C_true
P = K @ np.hstack([R, t[:, None]])              # 3x4 camera matrix

def project(P, X):
    x = np.hstack([X, np.ones((len(X), 1))]) @ P.T
    return x[:, :2] / x[:, 2:]

X = np.array([[0.0, 0.0, 0.0], [1.0, 0.0, 0.0], [0.0, 1.0, 1.0]])
print("pixels:\n", project(P, X))
print("depths Z_cam:", (X @ R.T + t)[:, 2])

# Camera centre: C = -R^T t, which is also the null vector of P
_, _, Vt = np.linalg.svd(P)
print("C = -R^T t:", -R.T @ t, " null(P):", Vt[-1, :3] / Vt[-1, 3])

# Field of view from f and the image width
print("horizontal FOV: %.2f deg" % np.degrees(2 * np.arctan(1280 / (2 * fx))))

# Halve the resolution, then crop a window starting at pixel (100, 50)
S = np.diag([0.5, 0.5, 1.0])
K_half = S @ K
K_crop = np.array([[1, 0, -100], [0, 1, -50], [0, 0, 1]]) @ K_half
print("K at 640 x 480:\n", K_half)
print("K after crop:\n", K_crop)
print("halved pixels match:", np.allclose(project(S @ P, X), 0.5 * project(P, X)))

def decompose(P):
    """Split P = K [R | t] using an RQ decomposition of its left 3x3 block."""
    K, R = rq(P[:, :3])
    D = np.diag(np.sign(np.diag(K)))            # force a positive diagonal on K
    K, R = K @ D, D @ R                         # D @ D = I, so K @ R is unchanged
    if np.linalg.det(R) < 0:                    # P is known only up to sign
        K, R = -K, -R
    t = np.linalg.solve(K, P[:, 3])
    return K / K[2, 2], R, t

for c in (1.0, -3.0):                           # any nonzero scale of P
    K_est, R_est, t_est = decompose(c * P)
    print(f"scale {c}: K =\n", np.round(K_est, 6) + 0.0)
    print("  R ok:", np.allclose(R_est, R), " t ok:", np.allclose(t_est, t))

Output:

pixels:
 [[125.966 453.64 ]
 [465.418 465.616]
 [169.255 716.709]]
depths Z_cam: [5.911 5.998 6.99 ]
C = -R^T t: [ 1.  -0.5 -6. ]  null(P): [ 1.  -0.5 -6. ]
horizontal FOV: 35.49 deg
K at 640 x 480:
 [[1000.    0.  320.]
 [   0. 1000.  240.]
 [   0.    0.    1.]]
K after crop:
 [[1000.    0.  220.]
 [   0. 1000.  190.]
 [   0.    0.    1.]]
halved pixels match: True
scale 1.0: K =
 [[2000.    0.  640.]
 [   0. 2000.  480.]
 [   0.    0.    1.]]
  R ok: True  t ok: True
scale -3.0: K =
 [[2000.    0.  640.]
 [   0. 2000.  480.]
 [   0.    0.    1.]]
  R ok: True  t ok: True

The translation $\mathbf{t}$ is not the camera position; $-R^\top\mathbf{t}$ is, and it agrees with the null vector of $P$. The decomposition recovers $K$, $R$, and $\mathbf{t}$ even from $-3P$: the sign fix makes $K$ and $R$ come out the same, and dividing by $K$’s bottom-right entry and solving with the unnormalized $K$ cancel the scale.

Common Misconceptions

• “$\mathbf{t}$ is where the camera is.” In the world-to-camera convention, $\mathbf{t}$ is the world origin in camera coordinates. The camera center is $-R^\top\mathbf{t}$.
• “The focal length on the lens is $f_x$.” $f_x$ is in pixels and depends on the pixel pitch and on any resizing of the image. A 4 mm lens on 2 µm pixels gives 2000 pixels, which becomes 1000 if the image is downsampled by two.
• “The principal point is the image center.” It is close to the center but usually offset by some pixels, and assuming the center is a frequent source of systematic error in metric work.
• “$K$ is a property of the camera alone.” It is a property of the camera at a given focus, zoom, and image resolution. Refocusing, zooming, digital cropping, or resizing all change it.
• “Every pose matrix means the same thing.” A $4 \times 4$ matrix may be world-to-camera or camera-to-world, and its camera axes may follow the vision or the graphics convention. The matrix itself does not say which.

Where It Is Used

Intrinsics and extrinsics are estimated by camera calibration, which observes a known target, usually a planar pattern, in several views and fits all parameters, including lens distortion, by minimizing reprojection error. Once $K$ is known, two-view geometry can be reduced to the essential matrix in normalized coordinates, and points seen in several calibrated views can be triangulated. Perspective-n-Point estimates the extrinsics of a calibrated camera from known 3D points, which is how augmented reality anchors virtual content and how visual localization works. Structure from motion and SLAM estimate the extrinsics of every frame, and sometimes the intrinsics too, by bundle adjustment; filtering approaches such as the extended Kalman filter track camera pose over time. Multi-camera rigs on vehicles and robots store each camera’s intrinsics together with its extrinsics relative to the rig, and rendering pipelines use the same parameters to place a virtual camera.