Geometry & 3D Vision
Camera Pose
The position and orientation of a camera as a rigid transform in SE(3), the two directions it is stored in (world-to-camera and camera-to-world), the axis conventions of OpenCV, COLMAP, OpenGL and NeRF, and how pose errors are measured.
intermediate
A camera’s pose is where it is and which way it points: the rigid motion that relates the camera’s own coordinate frame to the world. It is the extrinsic half of the camera parameters, and the quantity that perspective-n-point, structure from motion, visual odometry and SLAM estimate and neural renderers consume. A pose is only six numbers, but it is stored in two directions and with several axis conventions, and a mismatch in either produces cameras that look away from the scene or see it mirrored, usually without an error message.
Definition
The pose of a camera is a rigid transformation, a rotation and a translation , between world coordinates and camera coordinates. It has six degrees of freedom: three for orientation and three for position. Rigid transforms preserve distances and handedness and form the special Euclidean group .
The pose is distinct from the intrinsics , which map directions in the camera frame to pixels and do not change when the camera moves. A pose says nothing about focal length or image size; the intrinsics say nothing about where the camera is.
As in the camera parameters article, the extrinsics map world to camera, ; this article calls the inverse, camera-to-world transform the pose matrix. How itself is parameterized is the subject of rotation representations.
Intuition
Two questions can be asked of a camera. “Where is this world point relative to me?” is answered by the world-to-camera transform, which projection needs. “Where am I, and where am I looking?” is answered by the camera-to-world transform, whose translation is the camera’s position and whose rotation columns are the camera’s axes drawn in the world. Vision code tends to store the first, robotics and graphics code the second.
A second, independent choice is how the camera’s axes are labelled: vision cameras look along with down, graphics cameras along with up.
Formal Definition
The two directions
As homogeneous matrices, with read “world to camera”,
The camera centre is , not . The columns of (the rows of ) are the camera’s , and axes in world coordinates. The viewing direction in the world is therefore the third column of for a camera that looks along , and its negative for one that looks along .
Naming and composition
A subscript convention keeps chains of transforms honest: maps coordinates in frame to frame , so inner indices cancel, , and . OpenCV’s documentation writes the same idea as , the transform from an object frame to the camera frame, and chains ; COLMAP names its rig poses RIG_FROM_WORLD. Neither “pose” nor a bare says which direction is meant; a name of the form cam_from_world or does.
Relative pose
For two cameras with extrinsics and , the transform from camera-1 coordinates to camera-2 coordinates is
This is the of the essential matrix, , with camera 1 placed at the origin. Its translation has length , the baseline, which two images alone determine only up to scale.
Axis conventions
The vision and graphics camera frames are related by
a rotation of 180° about that is its own inverse. It changes only the camera frame, so the camera centre is unchanged. As of October 2026, the official documentation of the common tools states:
| Source | Camera axes | What is stored |
|---|---|---|
OpenCV 4.14.0 calib3d (same text in 5.0.0 calib) |
right, down, forward (pinhole figure) | , (as rvec, tvec) change coordinates from world to camera |
COLMAP 4.2.1 images.txt |
right, down, forward | world-to-camera, QW QX QY QZ TX TY TZ, Hamilton quaternion; centre |
OpenGL gluLookAt |
right, up, looking along | view matrix, world-to-camera |
| NeRF code (bmild/nerf) | right, up, backward (“as in OpenGL”) | camera-to-world matrices |
LLFF poses_bounds.npy |
columns | camera-to-world, plus height, width, focal length |
| TUM RGB-D ground truth | not stated (pose of the colour camera’s optical centre) | tx ty tz qx qy qz qw: optical centre and orientation in the world |
COLMAP’s format page states the world-to-camera direction, the quaternion order and the camera centre formula explicitly. The NeRF repository’s README states both the OpenGL axes and the camera-to-world direction, and its LLFF loader reorders the LLFF columns into right, up, backwards before use. Nerfstudio’s documentation calls the same frame the “OpenGL/Blender (and original NeRF)” convention and notes that COLMAP/OpenCV flips the and axes. The TUM benchmark stores the camera’s position and orientation in the world, a camera-to-world pose, with the quaternion scalar last.
Properties
Estimating pose
- From known 3D points: perspective-n-point solves for of a calibrated camera from 2D–3D matches.
- Between two views: the essential matrix gives the relative pose with the translation up to scale; structure from motion chains and refines such estimates over many images with bundle adjustment.
- Between point clouds: iterative closest point aligns depth scans and returns the rigid motion between the sensor poses.
- Over time: visual odometry chains frame-to-frame motions, and SLAM adds loop closures to correct the drift.
Optimizers update a pose with small increments in the tangent space of rather than editing matrix entries; rotation representations covers that machinery for the rotation part.
Measuring pose error
Single poses. The rotation error is the angle of , and the translation error is best taken between camera centres, , as in the PnP article’s example. Comparing directly mixes in the rotation error, scaled by the distance to the world origin. The two are reported separately because they have different units and no natural exchange rate: whether 1° is worse than 5 cm depends on the scene’s depth. When the translation is known only up to scale, as from an essential matrix, its error is an angle between directions.
Trajectories. Sturm et al. (2012) defined the two metrics of their TUM RGB-D benchmark for estimated poses and ground truth in , both camera-to-world. The relative pose error over an interval is
summarized as the RMS of its translational part (their Eq. 2); gives the drift per frame, and for SLAM they average over all . The absolute trajectory error first aligns the two trajectories, because each may live in its own coordinate frame, with the least-squares rigid transform given in closed form by Horn (1987), then takes the RMS of the translational part of . RPE measures local drift; ATE measures global consistency. It follows from the definition that RPE needs no alignment, since does not change when the whole trajectory is expressed in another frame.
A monocular trajectory also has an arbitrary scale, so monocular results are aligned with a similarity transform, Sim(3), as the visual odometry article describes for DSO’s EuRoC results; Horn’s closed form includes the scale factor. Aligning with more freedom than the sensor leaves undetermined hides real errors: a stereo or RGB-D system has metric scale, and Sim(3) alignment would forgive a scale error it should be charged for.
Interpolating poses
Smooth camera paths and trajectories sampled at different rates need poses between keyframes. Slerp on the rotation (rotation representations) with linear interpolation of the centre treats the two independently; the exponential, , moves them together. Solà et al. (2018, arXiv v9, Example 7) contrast these parameterizations. Interpolating the twelve matrix entries does not give a rigid transform.
Examples
The first script builds three cameras looking at a cube with a look-at helper, converts their poses between directions and axis conventions, checks the OpenGL pose against the gluLookAt construction, and then makes three common mistakes.
import numpy as np
np.set_printoptions(precision=3, suppress=True)
def look_at_cv(C, target, up=(0.0, 0.0, 1.0)):
"""World-to-camera (R, t) for an OpenCV/COLMAP camera: x right, y down, z forward."""
z = (target - C) / np.linalg.norm(target - C)
x = np.cross(z, up); x /= np.linalg.norm(x)
y = np.cross(z, x)
R = np.stack([x, y, z]) # rows = camera axes in world coordinates
return R, -R @ C # X_c = R X_w + t
def to_4x4(R, t):
T = np.eye(4); T[:3, :3] = R; T[:3, 3] = t
return T
def project(T_cw, K, X):
"""Project world points with a world-to-camera transform; return pixels and depths z_c."""
Xc = X @ T_cw[:3, :3].T + T_cw[:3, 3]
uv = Xc @ K.T
return uv[:, :2] / uv[:, 2:], Xc[:, 2]
# Scene: the 8 corners of a 1 m cube at the world origin; world z is up.
X = np.array([[i, j, k] for i in (-1, 1) for j in (-1, 1) for k in (-1, 1)], float) * 0.5
K = np.array([[800, 0, 320], [0, 800, 240], [0, 0, 1.0]])
centres = [np.array([4.0, 0.0, 1.5]), np.array([3.0, 3.0, 2.0]), np.array([0.0, 4.5, 1.0])]
T_cw = [to_4x4(*look_at_cv(C, np.zeros(3))) for C in centres] # world-to-camera (extrinsics)
for i, T in enumerate(T_cw):
R, t = T[:3, :3], T[:3, 3]
uv, z = project(T, K, X)
print(f"cam {i}: t = {t}, C = -R^T t = {-R.T @ t}, points with z > 0: {np.sum(z > 0)}/8")
uv_true, _ = project(T_cw[0], K, X)
print("cam 0 pixels of the first 3 corners:\n", uv_true[:3])
# Camera-to-world (the pose matrix): its translation is the centre and its
# third column is the viewing direction of an OpenCV camera.
T_wc = np.linalg.inv(T_cw[0])
print("T_wc translation:", T_wc[:3, 3], "| viewing direction (3rd column):", T_wc[:3, 2])
print("unit vector from C to the cube centre:", -centres[0] / np.linalg.norm(centres[0]))
# OpenGL/NeRF camera axes (x right, y up, z backward): flip the y and z axes.
F = np.diag([1.0, -1.0, -1.0, 1.0])
T_wc_gl = T_wc @ F # camera-to-world, OpenGL axes
print("OpenGL pose: viewing direction = -(3rd column):", -T_wc_gl[:3, 2])
# Check against the gluLookAt construction (Khronos reference page): rows s, u, -f.
f = -centres[0] / np.linalg.norm(centres[0]); up = np.array([0.0, 0.0, 1.0])
s = np.cross(f, up); u = np.cross(s / np.linalg.norm(s), f)
M = np.eye(4); M[:3, :3] = np.stack([s / np.linalg.norm(s), u, -f])
view = M @ to_4x4(np.eye(3), -centres[0]) # gluLookAt = M * translate(-eye)
print("inverse of the gluLookAt view matrix equals T_wc_gl:", np.allclose(np.linalg.inv(view), T_wc_gl))
# Three mistakes, each tested on all three cameras with the OpenCV projection.
Fz = np.diag([1.0, 1.0, -1.0, 1.0])
cases = {"correct world-to-camera": lambda T: T,
"camera-to-world used as world-to-camera": lambda T: np.linalg.inv(T),
"OpenGL pose, flip forgotten": lambda T: np.linalg.inv(np.linalg.inv(T) @ F),
"OpenGL pose, only z negated (det = -1)": lambda T: np.linalg.inv(np.linalg.inv(T) @ F @ Fz)}
for name, wrong in cases.items():
res = []
for T in T_cw:
uv_ok, _ = project(T, K, X)
uv, z = project(wrong(T), K, X)
res.append(f"{np.sum(z > 0)}/8 {np.linalg.norm(uv - uv_ok, axis=1).mean():7.1f}")
print(f"{name:40s} z > 0, mean px error:", " | ".join(res))
uv, _ = project(cases["OpenGL pose, only z negated (det = -1)"](T_cw[0]), K, X)
print("only z negated: max |u - u_true| = %.1e, max |v - (2 c_y - v_true)| = %.1e"
% (np.abs(uv[:, 0] - uv_true[:, 0]).max(), np.abs(uv[:, 1] - (480 - uv_true[:, 1])).max()))
Output:
cam 0: t = [0. 0. 4.272], C = -R^T t = [4. 0. 1.5], points with z > 0: 8/8
cam 1: t = [0. 0. 4.69], C = -R^T t = [3. 3. 2.], points with z > 0: 8/8
cam 2: t = [0. 0. 4.61], C = -R^T t = [0. 4.5 1. ], points with z > 0: 8/8
cam 0 pixels of the first 3 corners:
[[238.629 287.619]
[232.369 127.179]
[401.371 287.619]]
T_wc translation: [4. 0. 1.5] | viewing direction (3rd column): [-0.936 0. -0.351]
unit vector from C to the cube centre: [-0.936 -0. -0.351]
OpenGL pose: viewing direction = -(3rd column): [-0.936 -0. -0.351]
inverse of the gluLookAt view matrix equals T_wc_gl: True
correct world-to-camera z > 0, mean px error: 8/8 0.0 | 8/8 0.0 | 8/8 0.0
camera-to-world used as world-to-camera z > 0, mean px error: 8/8 2427.3 | 8/8 1820.8 | 8/8 4884.6
OpenGL pose, flip forgotten z > 0, mean px error: 0/8 189.9 | 0/8 120.9 | 0/8 175.6
OpenGL pose, only z negated (det = -1) z > 0, mean px error: 8/8 177.2 | 8/8 155.4 | 8/8 171.2
only z negated: max |u - u_true| = 5.7e-14, max |v - (2 c_y - v_true)| = 1.1e-13
For cameras looking at the origin, : the world origin sits straight ahead at distance , while the centres are elsewhere. The pose matrix’s third column is the viewing direction, and the flipped OpenGL pose equals the inverse of the gluLookAt view matrix. Each mistake fails differently. Using the camera-to-world matrix for projection leaves every point in front of these cameras only because the wrong depth, , is each centre’s height (1–2 m) give or take at most the cube’s 0.87 m half-diagonal; yet the pixels are off by 1,800–4,900 px, and a “points in front” check would not catch it. Forgetting the flip puts every point behind the camera. Negating only is not a rotation (determinant ), and it yields a valid-looking image mirrored top to bottom about the principal point row.
The second script computes a relative pose, the errors of a perturbed pose, and the trajectory errors of a 60-pose loop estimated in a foreign frame with 2 cm and 0.3° of noise per pose, once with the correct scale (as from stereo) and once at 0.4 times the scale (as from a single camera).
import numpy as np
from scipy.spatial.transform import Rotation
def look_at_cv(C, target, up=(0.0, 0.0, 1.0)):
z = (target - C) / np.linalg.norm(target - C)
x = np.cross(z, up); x /= np.linalg.norm(x)
return np.stack([x, np.cross(z, x), z]), C # (R_cw, camera centre)
def skew(v):
return np.array([[0, -v[2], v[1]], [v[2], 0, -v[0]], [-v[1], v[0], 0]])
def angle_deg(Ra, Rb):
"""Geodesic angle of Ra^T Rb, atan2 form."""
M = Ra.T @ Rb
s = np.linalg.norm([M[2, 1] - M[1, 2], M[0, 2] - M[2, 0], M[1, 0] - M[0, 1]]) / 2
return np.degrees(np.arctan2(s, (np.trace(M) - 1) / 2))
X = np.array([[i, j, k] for i in (-1, 1) for j in (-1, 1) for k in (-1, 1)], float) * 0.5
R0, C0 = look_at_cv(np.array([4.0, 0.0, 1.5]), np.zeros(3))
R1, C1 = look_at_cv(np.array([3.0, 3.0, 2.0]), np.zeros(3))
t0, t1 = -R0 @ C0, -R1 @ C1
# Relative pose: camera-0 coordinates -> camera-1 coordinates, T_10 = T_1w T_w0.
R10 = R1 @ R0.T
t10 = t1 - R10 @ t0
Xc0, Xc1 = X @ R0.T + t0, X @ R1.T + t1
print("X_c1 = R_10 X_c0 + t_10:", np.allclose(Xc0 @ R10.T + t10, Xc1))
print("relative rotation %.2f deg, |t_10| = %.3f = |C1 - C0| = %.3f"
% (angle_deg(np.eye(3), R10), np.linalg.norm(t10), np.linalg.norm(C1 - C0)))
E = skew(t10) @ R10 # essential matrix for this pair
x0, x1 = Xc0 / Xc0[:, 2:], Xc1 / Xc1[:, 2:] # normalized image coordinates
print("epipolar residuals max |x1^T E x0| = %.1e" % np.abs(np.einsum("ij,jk,ik->i", x1, E, x0)).max())
# Pose error of a perturbed estimate of camera 1: 2 deg about an arbitrary axis, centre 5 cm off.
dR = Rotation.from_rotvec(np.radians(2.0) * np.array([1.0, 2.0, 2.0]) / 3).as_matrix()
R1_est = dR @ R1
C1_est = C1 + np.array([0.03, 0.0, -0.04])
t1_est = -R1_est @ C1_est
print("rotation error %.3f deg | centre error %.3f m | |t_est - t| = %.3f m"
% (angle_deg(R1, R1_est), np.linalg.norm(C1_est - C1), np.linalg.norm(t1_est - t1)))
# Trajectory error: 60 poses on a loop, estimated in another frame (and, for a
# monocular system, another scale) with noise.
def align(P, Q, with_scale):
"""Least-squares Q ~ s R P + t (rotation by SVD; scale as in Horn 1987, sec. 2.D)."""
mp, mq = P.mean(0), Q.mean(0)
Pc, Qc = P - mp, Q - mq
U, _, Vt = np.linalg.svd(Qc.T @ Pc)
D = np.diag([1.0, 1.0, np.sign(np.linalg.det(U @ Vt))])
R = U @ D @ Vt
s = np.sum(Qc * (Pc @ R.T)) / np.sum(Pc ** 2) if with_scale else 1.0
return s, R, mq - s * R @ mp
def ate(P, Q, mode):
s, R, t = (1.0, np.eye(3), np.zeros(3)) if mode == "none" else align(P, Q, mode == "Sim(3)")
return np.sqrt(np.mean(np.sum((s * P @ R.T + t - Q) ** 2, axis=1)))
def rpe(Pw, Qw, Ps, Qs):
"""RMS translational RPE for Delta = 1 (Sturm et al. 2012, eqs. 1-2); poses are camera-to-world (R, C)."""
errs = []
for i in range(len(Ps) - 1):
dQ = (Qw[i].T @ Qw[i + 1], Qw[i].T @ (Qs[i + 1] - Qs[i])) # Q_i^-1 Q_{i+1}
dP = (Pw[i].T @ Pw[i + 1], Pw[i].T @ (Ps[i + 1] - Ps[i]))
errs.append(dQ[0].T @ (dP[1] - dQ[1])) # translation of dQ^-1 dP
return np.sqrt(np.mean(np.sum(np.array(errs) ** 2, axis=1)))
def trajectory_errors(seed):
rng = np.random.default_rng(seed)
a = np.linspace(0, 2 * np.pi, 60, endpoint=False)
Q = np.stack([3 * np.cos(a), 2 * np.sin(a), 1.5 + 0.2 * np.sin(3 * a)], 1) # true centres (m)
Qw = [look_at_cv(c, np.zeros(3))[0].T for c in Q] # R_wc
G = Rotation.from_euler("zyx", [70, -20, 10], degrees=True).as_matrix() # unknown frame change
out = {}
for name, scale in (("stereo", 1.0), ("monocular", 0.4)):
P = scale * (Q + rng.normal(0, 0.02, Q.shape)) @ G.T + np.array([5.0, -2.0, 1.0])
Pw = [G @ Rw @ Rotation.from_rotvec(rng.normal(0, np.radians(0.3), 3)).as_matrix() for Rw in Qw]
out[name] = [ate(P, Q, m) for m in ("none", "SE(3)", "Sim(3)")] + [rpe(Pw, Qw, P, Q)]
return out
for name, e in trajectory_errors(0).items():
print(f"seed 0, {name:9s}: ATE no alignment {e[0]:6.3f} m, SE(3) {e[1]:.3f} m, "
f"Sim(3) {e[2]:.3f} m | RPE (Delta = 1) {e[3]:.3f} m")
runs = [trajectory_errors(s) for s in range(10)]
for name in ("stereo", "monocular"):
v = np.array([r[name][2] for r in runs])
print(f"seeds 0-9, {name:9s}: Sim(3) ATE from {v.min():.3f} to {v.max():.3f} m")
Output:
X_c1 = R_10 X_c0 + t_10: True
relative rotation 45.23 deg, |t_10| = 3.202 = |C1 - C0| = 3.202
epipolar residuals max |x1^T E x0| = 4.9e-17
rotation error 2.000 deg | centre error 0.050 m | |t_est - t| = 0.164 m
seed 0, stereo : ATE no alignment 5.872 m, SE(3) 0.032 m, Sim(3) 0.032 m | RPE (Delta = 1) 0.048 m
seed 0, monocular: ATE no alignment 5.757 m, SE(3) 1.532 m, Sim(3) 0.035 m | RPE (Delta = 1) 0.164 m
seeds 0-9, stereo : Sim(3) ATE from 0.029 to 0.036 m
seeds 0-9, monocular: Sim(3) ATE from 0.031 to 0.037 m
The relative pose maps camera-0 points onto camera-1 points, its translation length is the baseline, and it satisfies the epipolar constraint to rounding error. For the perturbed camera, the centre is 5 cm off, but moves by 16.4 cm, because the 2° rotation error also swings the world origin, 4.69 m away, by up to m, depending on the rotation axis. Without alignment, the ATE measures only the frame change. Rigid alignment brings the stereo case down to the noise level (about 3 cm), while the monocular case needs the scale as well: 1.53 m with SE(3), 3.5 cm with Sim(3). Over seeds 0–9 the aligned ATE stays between 2.9 and 3.7 cm. The monocular RPE is computed in the estimate’s own units and is dominated by the scale error, so for a monocular estimate it means little until the scale is fixed.
Common Misconceptions
- “A 4×4 pose matrix is self-describing.” Nothing in the matrix says whether it maps world to camera or camera to world. If its translation is the camera’s position, it is camera-to-world; if is, it is world-to-camera.
- “Converting OpenCV poses to OpenGL means negating .” The conversion negates both and ; negating one axis is a reflection and mirrors the image. Whether multiplies on the right (camera-to-world) or on the left (world-to-camera) depends on the direction.
- “Angles are in degrees.” OpenCV’s
Rodriguestakes the norm of a rotation vector as the angle in radians, and SciPy’sfrom_eulerreads radians unlessdegrees=Trueis passed. Degrees read as radians give a large, wrong rotation; radians read as degrees give an almost unrotated camera. - “Translations are in metres.” Units vary between datasets and tools, and monocular reconstructions such as COLMAP’s have an arbitrary scale until they are aligned to something metric. Errors in such a frame mean nothing without stating the alignment.
- “ATE after Sim(3) alignment is always fair.” Only when the scale is genuinely unobservable; for metric sensors it hides scale error.
Where It Is Used
Every multi-view method needs camera poses. Perspective-n-point localizes a camera against a map, the essential matrix and triangulation relate two views, and structure from motion, visual odometry and SLAM estimate whole trajectories, evaluated by ATE and RPE. Multi-view stereo and depth fusion place each depth map in a common frame by its camera-to-world pose, and neural radiance fields and Gaussian splatting cast rays or project primitives from posed images, often with poses computed by COLMAP and converted to the OpenGL convention. Robot and vehicle rigs chain camera poses with sensor-to-body and body-to-world transforms, where the subscript convention prevents the most common errors.
Related
- Pinhole Camera Model
The ideal camera in which every scene point projects through a single center onto an image plane, giving the perspective projection x = fX/Z, y = fY/Z.
- Perspective-n-Point (PnP)
Estimating the position and orientation of a calibrated camera from known 3D points and their 2D image projections, the absolute pose problem behind visual localization, augmented reality, and camera tracking.
- Essential Matrix
The 3 × 3 matrix E = [t]×R that encodes the relative rotation and translation direction between two calibrated views, how it is estimated from point matches, and how the camera pose is recovered from it.
- Structure from Motion
Recovering the poses and intrinsics of the cameras that took a set of images, together with a sparse 3D point cloud of the scene, from the images alone, by incremental or global reconstruction.
- Visual Odometry
Estimating the motion of a camera, and of the vehicle or device carrying it, incrementally from a video stream, frame after frame, without building a globally consistent map.
- Simultaneous Localization and Mapping (SLAM)
Building a map of an unknown environment while estimating the trajectory of the sensor moving through it, and using recognized revisits to keep both globally consistent.
References
- Sturm, J., Engelhard, N., Endres, F., Burgard, W. & Cremers, D. (2012). A Benchmark for the Evaluation of RGB-D SLAM Systems. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 573–580.
- Horn, B. K. P. (1987). Closed-Form Solution of Absolute Orientation Using Unit Quaternions. Journal of the Optical Society of America A, 4(4), 629–642.
- Solà, J., Deray, J. & Atchuthan, D. (2018). A Micro Lie Theory for State Estimation in Robotics. arXiv:1812.01537 (version 9, 2021).
- COLMAP 4.2.1 documentation source (2026): Output Format, doc/format.rst.
- OpenCV 4.14.0 source code: camera model documentation in modules/calib3d/include/opencv2/calib3d.hpp (2026). The same text is in modules/calib/include/opencv2/calib.hpp of OpenCV 5.0.0.
- Khronos Group. gluLookAt. OpenGL 2.1 Reference Pages.
- Mildenhall, B. et al. NeRF official code repository (bmild/nerf): README, section Generating poses for your own scenes.
- Mildenhall, B. et al. LLFF official code repository (Fyusion/LLFF): README, section Using your own poses without running COLMAP.
- Nerfstudio documentation: Data conventions.
- TUM Computer Vision Group. RGB-D SLAM Dataset and Benchmark: File Formats.