Geometry & 3D Vision

ORB-SLAM

The feature-based visual SLAM system that uses ORB features for tracking, mapping, relocalization and loop closing in three parallel threads, with covisibility and Essential Graphs, automatic monocular initialization, Sim(3) loop correction, and its successors ORB-SLAM2 (stereo and RGB-D) and ORB-SLAM3 (visual-inertial and multi-map).

advanced

ORB-SLAM is a keyframe-based visual SLAM system that estimates a camera’s trajectory and a sparse map of 3D points from a single camera, in real time and without a GPU. Mur-Artal, Montiel and Tardós (2015) designed it around one choice: the same ORB features serve every task, tracking, mapping, relocalization and loop closing. Three threads run in parallel. Tracking localizes every frame against a local map, local mapping refines the neighbourhood of each new keyframe by bundle adjustment, and loop closing recognizes revisited places and removes the drift accumulated around the loop, including drift of the map’s scale. Two successors extend it: ORB-SLAM2 added stereo and RGB-D cameras and a full bundle adjustment after each loop closure, and ORB-SLAM3 added visual-inertial estimation and a set of maps that are merged when the camera returns to a mapped area.

The SLAM article places ORB-SLAM among the keyframe-based systems that followed PTAM; this article describes how it works. The 2015 paper is read here in the authors’ accepted manuscript. The ORB-SLAM2 and ORB-SLAM3 journal papers could not be read for this article; those sections follow the PhD theses in which their authors describe the systems (Mur Artal, 2017; Campos Martínez, 2021; Elvira López-Echazarreta, 2024) and the official code at pinned commits.

Problem

Monocular SLAM must build a map from bearings alone. Depth cannot be recovered from one image, so the system needs a separate procedure to create its first map, and the map and trajectory are known only up to an unknown scale that can drift as the camera explores (Mur-Artal et al., 2015, §II-B–C; Strasdat et al., 2010). Keyframe-based systems keep bundle adjustment affordable by optimizing only selected frames. Mur-Artal et al. (2015, §I) list what such a system has to supply to bundle adjustment in real time: corresponding observations of map points across keyframes, a keyframe selection that avoids redundancy, a strong network of keyframes with wide parallax and loop closure matches, initial estimates for the optimization, a local window for exploration, and fast global optimization to close loops.

PTAM, the starting point, split tracking and mapping into two threads, but the ORB-SLAM authors list its limits: no loop closing, poor handling of occlusions, relocalization that tolerates little change of viewpoint, and a map that has to be bootstrapped by hand (2015, §I). ORB-SLAM keeps PTAM’s main ideas and adds the place recognition of Gálvez-López and Tardós (2012), the scale-aware loop closing of Strasdat et al. (2010) and the use of covisibility to keep tracking and mapping local.

Inputs and Outputs

  • Input: a stream of images from a calibrated camera (focal length, principal point and, optionally, a distortion model; keypoint coordinates are undistorted), plus an ORB vocabulary trained offline. ORB-SLAM2 also takes rectified stereo pairs or registered RGB-D frames; ORB-SLAM3 also takes inertial measurements and supports pinhole and fisheye lens models.
  • Output: a pose for every frame, keyframe poses Tiw∈SE(3)T_{iw} \in SE(3) that map world points into the camera frame, and a map of 3D points. For a monocular camera the map has an arbitrary scale, so trajectories are compared with ground truth after a similarity alignment (Mur-Artal et al., 2015, §VIII-B).

Each map point stores its 3D position, its mean viewing direction, one representative ORB descriptor (the one with the smallest Hamming distance to the point’s other observations), and the range of distances over which ORB’s scale invariance lets it be observed. Each keyframe stores its pose, the camera intrinsics and all ORB features extracted in it, matched to map points or not (2015, §III-C).

Intuition

Two ideas carry most of the design. The first is that recognizable features make the whole map reusable. Because each map point carries an ORB descriptor, a frame can be matched to keyframes seen minutes earlier, by place recognition instead of by tracking, so the same machinery relocalizes a lost camera, closes loops and resumes work in a mapped area.

The second is that the map should be local wherever possible. Keyframes that see the same map points are linked in a covisibility graph. Tracking only searches the map points of keyframes near the current frame in that graph, and local bundle adjustment only moves those keyframes, so the cost of each step depends on the size of the local neighbourhood rather than of the whole map. When a loop is closed, the system optimizes a sparse subgraph, the Essential Graph, instead of the full map.

A third idea, that the system should be “generous in the spawning but very restrictive in the culling” (Mur-Artal et al., 2015, §I), governs map management: keyframes and points are created freely, which keeps tracking alive through fast motion, and redundant ones are deleted later, so the map grows with the scene rather than with time.

Algorithm

Map structures

The covisibility graph has a node per keyframe and an edge between two keyframes that share observations of at least 15 map points, weighted by the number θ\theta of shared points (2015, §III-D; int th = 15 in the code’s KeyFrame::UpdateConnections). It can be dense, so the system also maintains a spanning tree: each new keyframe is linked to the keyframe with which it shares most observations. The Essential Graph keeps all keyframes but only the spanning tree, the covisibility edges with θ≥θmin=100\theta \ge \theta_{min} = 100, and the loop closure edges. On the NewCollege sequence the authors report that it has about five times as many edges as keyframes (2015, §VIII-A).

Place recognition uses DBoW2 (Gálvez-López and Tardós, 2012), a bag of binary words: an ORB vocabulary built offline discretizes the descriptor space, and an inverted index stores, for each word, the keyframes in which it appears. ORB-SLAM groups database matches by covisibility rather than by time, returns all keyframes scoring above 75% of the best, and speeds up feature matching by comparing only features that fall under the same vocabulary node at the second of six levels (2015, §III-E). The general design of loop detection and of place-recognition indexes belongs to loop closure and to visual place recognition; what follows are ORB-SLAM’s own choices.

Initialization

A monocular map starts from two frames with enough parallax. ORB-SLAM computes two models in parallel inside RANSAC, a homography for a planar scene (normalized DLT on four points) and a fundamental matrix for a general one (eight-point algorithm on eight points), using the same number of iterations and the same point sets for both (2015, §IV):

  1. Extract ORB at the finest scale in the current frame and match it to a reference frame; reset the reference if there are too few matches.
  2. Score each model MM by its symmetric transfer errors d2d^2 in both directions, summing ρM(d2)=Γ−d2\rho_M(d^2) = \Gamma - d^2 for inliers (d2<TMd^2 < T_M) and 0 otherwise. The thresholds come from a χ2\chi^2 test at 95% for a one-pixel standard deviation, TH=5.99T_H = 5.99 and TF=3.84T_F = 3.84, and Γ=TH\Gamma = T_H, so that both models score the same for the same error.
  3. Select the homography if RH=SH/(SH+SF)>0.45R_H = S_H / (S_H + S_F) > 0.45, which captures planar and low-parallax scenes; otherwise select the fundamental matrix.
  4. Recover the motion hypotheses: eight from the homography (Faugeras and Lustman’s method), or four from the essential matrix E=K⊤FKE = K^\top F K. Triangulate every hypothesis and accept one only if it clearly wins: most points with parallax, in front of both cameras and with low reprojection error. If none wins, wait for more frames.
  5. Refine the initial map with a full bundle adjustment.

Refusing to initialize without a clear winner is what protects the system from low parallax and from the twofold ambiguity of a plane, which the authors call perhaps the key to the method’s robustness (2015, §IV). The released code does not use the paper’s 0.45: the ORB-SLAM2 code selects the homography for RH>0.40R_H > 0.40, with a comment giving the range 0.40–0.45, and the ORB-SLAM3 code for RH>0.50R_H > 0.50, with 0.400.40 commented out (Initializer.cc; TwoViewReconstruction.cc). Stereo and RGB-D cameras skip this step: the first keyframe is placed at the origin and its stereo points form the initial map (Mur Artal, 2017, §6.1.2).

Tracking

For every frame (2015, §V):

  1. Extract ORB at 8 pyramid levels with a scale factor of 1.2, spread over a grid; the ORB article gives the feature counts.
  2. Predict the pose with a constant-velocity model and search for the map points of the last frame near their predicted positions, with a wider search if too few are found; then optimize the pose alone (motion-only bundle adjustment).
  3. If tracking is lost, relocalize (below).
  4. Track the local map. The local map contains the keyframes K1\mathcal{K}_1 that share map points with the frame and their covisibility neighbours K2\mathcal{K}_2. Each of their map points is projected into the frame and discarded if it falls outside the image, if the angle between the current viewing ray and the point’s mean viewing direction exceeds 60°, or if its distance lies outside the point’s scale-invariance range; the distance also predicts the pyramid level at which to look for the match. The pose is then optimized again with all matches.
  5. Decide on a new keyframe. The paper requires that more than 20 frames have passed since the last relocalization, that local mapping is idle or more than 20 frames have passed since the last keyframe, that the frame tracks at least 50 points, and that it tracks less than 90% of the points of the reference keyframe. The last condition is a minimum visual change, used instead of PTAM’s distance to other keyframes. The ORB-SLAM2 code ties these frame counts to the camera’s frame rate.

Local mapping

For every new keyframe KiK_i (2015, §VI):

  1. Insert it into the covisibility graph and the spanning tree, and compute its bag-of-words vector.
  2. Cull recent map points. In the three keyframes after its creation, a new point must be found in more than 25% of the frames in which it is predicted to be visible and, once more than one keyframe has passed, be observed from at least three keyframes. Later it is removed only if fewer than three keyframes observe it.
  3. Create new points by triangulating unmatched ORB features against covisible keyframes, matching through the vocabulary, checking the epipolar constraint, and accepting a point only with positive depth in both views, enough parallax, low reprojection error and consistent scale.
  4. Run local bundle adjustment over KiK_i, its covisible keyframes and all their points; other keyframes that see those points enter the cost but stay fixed.
  5. Cull keyframes whose map points are 90% seen by at least three other keyframes at the same or a finer scale.

Loop closing

For each keyframe processed by local mapping (2015, §VII):

  1. Detect candidates. Compute the lowest bag-of-words similarity smins_{min} between KiK_i and its covisibility neighbours (θ≥30\theta \ge 30), query the database, and discard keyframes scoring below smins_{min} or directly connected to KiK_i. A candidate is accepted only after three consecutive consistent detections, consistency meaning that the candidates are connected in the covisibility graph.
  2. Compute a similarity. Match ORB features of map points between KiK_i and each candidate, giving 3D–3D correspondences. RANSAC with Horn’s closed-form absolute orientation finds a similarity SilS_{il} that maps coordinates in the frame of the loop keyframe KlK_l into that of KiK_i (the paper calls it the similarity from KiK_i to KlK_l); it is refined by minimizing reprojection errors in both images, extended by a guided search for more matches, and accepted if enough inliers support it. This step is also the loop’s geometric verification. It estimates a similarity because a monocular map can drift in seven degrees of freedom: three of rotation, three of translation and the scale (Strasdat et al., 2010).
  3. Fuse the loop. Correct the pose of KiK_i with SilS_{il} and propagate the correction to its neighbours, so that both sides of the loop align; project the map points of KlK_l and its neighbours into KiK_i and its neighbours, fuse duplicates, and update the covisibility edges, which now span the loop.
  4. Optimize the Essential Graph over Sim(3)Sim(3) to spread the loop error over the keyframes, then move each map point with the correction of one keyframe that observes it.

ORB-SLAM2 runs a full bundle adjustment after this, in a fourth thread (see ORB-SLAM2 below).

Relocalization

When tracking is lost, the frame is converted to a bag of words and the database is queried for candidate keyframes. For each candidate, ORB features associated with map points are matched, and RANSAC iterations are run in turn for each keyframe with EPnP, a perspective-n-point solver. A pose with enough inliers is optimized, more matches are searched with the candidate’s map points, and the pose is optimized again; if enough inliers support it, tracking resumes (2015, §V-C). In ORB-SLAM2’s code a candidate needs 15 matches to enter RANSAC and 50 inliers after the guided search to be accepted (Tracking::Relocalization).

Mathematical Formulation

All optimizations use Levenberg–Marquardt in g2o (2015, §III-B), a library described with factor graphs. Poses Tiw∈SE(3)T_{iw} \in SE(3) map world points into camera ii; with rotation RiwR_{iw} and translation tiw\mathbf t_{iw}, a world point Xw,j\mathbf X_{w,j} has camera coordinates (x,y,z)⊤=RiwXw,j+tiw(x, y, z)^\top = R_{iw}\mathbf X_{w,j} + \mathbf t_{iw} and projects to πi=(fux/z+cu, fvy/z+cv)\pi_i = (f_u x/z + c_u,\ f_v y/z + c_v).

Bundle adjustment minimizes the robust reprojection error over keyframe poses and map points (2015, Appendix):

C=∑i,jρh ⁣(ei,j⊤Ωi,j−1ei,j),ei,j=xi,j−πi(Tiw,Xw,j),C = \sum_{i,j} \rho_h\!\left(\mathbf e_{i,j}^\top \Omega_{i,j}^{-1} \mathbf e_{i,j}\right), \qquad \mathbf e_{i,j} = \mathbf x_{i,j} - \pi_i(T_{iw}, \mathbf X_{w,j}),

where ρh\rho_h is the Huber function and Ωi,j=σi,j2I2×2\Omega_{i,j} = \sigma_{i,j}^2 I_{2\times 2} depends on the pyramid level at which keypoint xi,j\mathbf x_{i,j} was detected. Full bundle adjustment fixes only the first keyframe; local bundle adjustment fixes the outer keyframes; motion-only bundle adjustment fixes all points. In the ORB-SLAM2 code the Huber thresholds are 5.991\sqrt{5.991} for monocular and 7.815\sqrt{7.815} for stereo observations, the 95% points of χ2\chi^2 with two and three degrees of freedom, and motion-only optimization runs four rounds of 10 iterations, reclassifying outliers between rounds.

Essential Graph optimization treats each keyframe pose as a similarity Siw∈Sim(3)S_{iw} \in Sim(3). Each edge between keyframes ii and jj has the residual

ei,j=log⁡Sim(3) ⁣(Sij Sjw Siw−1)∈R7,C=∑i,jei,j⊤Λi,j ei,j,\mathbf e_{i,j} = \log_{Sim(3)}\!\left(S_{ij}\, S_{jw}\, S_{iw}^{-1}\right) \in \mathbb R^7, \qquad C = \sum_{i,j} \mathbf e_{i,j}^\top \Lambda_{i,j}\, \mathbf e_{i,j},

where SijS_{ij} is the relative transformation computed from the SE(3)SE(3) poses just before the optimization with its scale set to 1, except on the loop edge, where it is the similarity from Horn’s method. The information matrices Λi,j\Lambda_{i,j} are set to the identity, and the loop keyframe is fixed to remove the seven gauge freedoms (2015, Appendix). The authors describe the method as a rough approximation of full bundle adjustment that converges faster and better in their experiments. Afterwards each similarity [sR∣t][sR \mid \mathbf t] is converted back to a rigid pose [R∣t/s][R \mid \mathbf t/s], which keeps the camera centre, and map points follow their reference keyframe (Optimizer::OptimizeEssentialGraph). The construction follows Strasdat et al. (2010, §IV-B), who converted every pose and relative-pose constraint to a similarity of scale 1, gave only the loop constraint its measured scale change, and optimized over the 7-vectors of sim(3)\mathfrak{sim}(3); the geometric transformations article derives the Sim(3)Sim(3) exponential and the gauge freedom of a monocular map.

Relative Sim(3)Sim(3) refinement for a loop candidate minimizes the reprojection error of the matched points in both keyframes, e1=x1,i−π1(S12,X2,j)\mathbf e_1 = \mathbf x_{1,i} - \pi_1(S_{12}, \mathbf X_{2,j}) and e2=x2,j−π2(S12−1,X1,i)\mathbf e_2 = \mathbf x_{2,j} - \pi_2(S_{12}^{-1}, \mathbf X_{1,i}), with Huber costs and the points held fixed (2015, Appendix).

Three implementation details are absent from the paper or differ from it. The released Essential Graph optimization runs 20 iterations (the paper’s Table VI experiments ran 10), starts Levenberg–Marquardt with a damping of 10−1610^{-16}, and receives a flag that fixes the scale for stereo and RGB-D input (OptimizeEssentialGraph, bFixScale). ORB-SLAM2’s code fixes the loop keyframe, as the paper says, while ORB-SLAM3’s fixes the map’s initial keyframe; the gauge choice does not change the result after alignment, as the example below checks. Horn’s method in the code (Sim3Solver::ComputeSim3) computes the rotation from the eigenvector of Horn’s 4×44 \times 4 matrix and the scale as ∑r1⋅Rr2/∑∥Rr2∥2\sum \mathbf r_{1}\cdot R\mathbf r_{2} / \sum \lVert R\mathbf r_{2}\rVert^2 over centred points, the asymmetric scale that the Kabsch algorithm article compares with Horn’s symmetric one.

Parameters

Parameter Value Source
ORB pyramid 8 levels, scale factor 1.2 2015, §V-A
Features per image 1,000 (512 × 384 to 752 × 480); 2,000 for KITTI 2015, §V-A
Covisibility edge at least 15 shared map points 2015, §III-D
Essential Graph covisibility edges θmin=100\theta_{min} = 100 2015, §III-D
Loop query reference neighbours with θ≥30\theta \ge 30 2015, §VII-A
Loop acceptance 3 consecutive consistent candidates 2015, §VII-A
Database returns scores above 75% of the best 2015, §III-E
Initialization thresholds TH=5.99T_H = 5.99, TF=3.84T_F = 3.84; homography if RH>0.45R_H > 0.45 2015, §IV
New keyframe > 20 frames since relocalization; ≥ 50 tracked points; < 90% of the reference keyframe’s points 2015, §V-E
Map point culling found in > 25% of predicted frames; ≥ 3 observing keyframes 2015, §VI-B
Keyframe culling 90% of its points seen by ≥ 3 other keyframes 2015, §VI-E
Close stereo point (ORB-SLAM2) depth < 40 × baseline Mur Artal, 2017, §6.1.1

The Essential Graph threshold trades time for density without much effect on accuracy. On KITTI sequence 09 the authors built the graph with θmin\theta_{min} of 200, 100, 50 and 15, giving 890 to 6,663 edges; the keyframe trajectory RMSE stayed between 8.36 and 8.95 m while the optimization time rose from 0.38 to 0.94 s (2015, Table VI).

Complexity

Tracking and local mapping are bounded by the size of the local covisibility neighbourhood, not by the size of the map. On the 2.2 km NewCollege sequence, recorded with a stereo camera at 20 fps and 512 × 382 pixels, on an Intel Core i7-4700MQ, tracking took a median of 30.57 ms per frame: 11.10 ms for ORB extraction, 3.38 ms for the initial pose and 14.84 ms for tracking the local map. Local mapping took a median of 383.59 ms per keyframe, most of it local bundle adjustment at 296.08 ms (2015, Table I). Loop detection grew sublinearly with the number of keyframes, which the authors attribute to the database comparing only images that share words. Over six loop closures with 287 to 4,496 keyframes, the total correction time grew from 0.51 to 4.69 s, of which Essential Graph optimization took 0.26 to 3.62 s (2015, Table II).

Running full bundle adjustment from the drifted state, instead of optimizing the Essential Graph, is the expensive choice ORB-SLAM avoids. On KITTI 09, without loop closing the keyframe RMSE was 48.77 m; 20 iterations of full bundle adjustment took 14.64 s and gave 49.90 m, 100 iterations took 72.16 s and gave 18.82 m, while the Essential Graph with θmin=100\theta_{min} = 100 took 0.48 s and gave 8.36 m, and 20 iterations of full bundle adjustment after it brought the error to 7.22 m (2015, Table VI). The authors conclude that before loop correction the solution is so far from the optimum that bundle adjustment has convergence problems.

Implementation

The official code is open source: ORB-SLAM2 (raulmur/ORB_SLAM2) and ORB-SLAM3 (UZ-SLAMLab/ORB_SLAM3, version 1.0 of December 2021, released under GPLv3 as of November 2026, with a closed-source version available from the authors). The example below isolates one step, the correction of a monocular loop.

Example: Sim(3) against SE(3) loop correction of monocular scale drift

The script builds a closed 110.3 m route of two 30 m straights and two half-circles of radius 8 m, with a keyframe every metre and 3 m of overlap, so the last of the 114 keyframes is back near keyframe 3. About 3,300 landmarks stand 3–6 m to either side of the route, and a keyframe sees those 1–15 m ahead in a 90° horizontal field of view. Poses are world-to-camera, as in ORB-SLAM (xc=sR xw+tx_c = sR\,x_w + t), the camera’s zz axis points along the route, xx to the right and yy down, and the world’s zz axis is up. A landmark unseen for more than ten keyframes gets a new map point when it is seen again, so when the camera returns to the start it builds duplicates, as a real system does before it closes the loop.

The monocular estimate chains the true relative motions with 0.05° of rotation noise and 1 cm of translation noise per step, but scales each translation by a local map scale λk\lambda_k that drifts from 1 at the start. The sources leave the shape of the drift open, so the script runs two profiles that reach the same end ratio: a steady exponential drift, and a Gaussian random walk (standard deviation 0.03 in log⁡λ\log\lambda per step) pinned to that end ratio. The keyframes’ map points are their true camera coordinates times λk\lambda_k, with 1% depth-proportional noise.

The graph is built as ORB-SLAM builds the Essential Graph: a spanning tree linking each keyframe to the earlier one sharing most map points, covisibility edges with at least 100 shared points, and one loop edge from keyframe 3 to the last keyframe. ORB-SLAM also adds edges between the neighbours of the two loop keyframes after fusion; the example leaves them out. Edge measurements are the drifted relative poses with scale 1, the loop measurement comes from Horn’s method on the shared landmarks’ two map-point copies, information matrices are the identity, and the loop keyframe is fixed. The SE(3)SE(3) correction uses the same graph with the scale held at 1 and a rigid Horn fit. The script writes the residual as log⁡(Zji−1SjwSiw−1)\log(Z_{ji}^{-1} S_{jw} S_{iw}^{-1}), with ZjiZ_{ji} the measured transformation from camera ii to camera jj, which is the paper’s residual with Sij=Zji−1S_{ij} = Z_{ji}^{-1}. It parameterizes each pose by a left increment on the drifted estimate, Siw=exp⁡(ξi) Siw0S_{iw} = \exp(\boldsymbol\xi_i)\, S^0_{iw}, and optimizes the ξi\boldsymbol\xi_i with Levenberg–Marquardt for at most 20 iterations, as the released code does. The measures are the absolute trajectory error of the keyframe positions after a Sim(3)Sim(3) (Umeyama) alignment, as ORB-SLAM’s experiments align monocular trajectories, and the range of the local scale: each step’s aligned length divided by its true length.

import numpy as np
from scipy.sparse import csr_matrix, diags
from scipy.sparse.linalg import spsolve
from scipy.spatial.transform import Rotation

# --- Sim(3) elements as (s, R, t), acting as x -> s R x + t; batched over a leading axis ---
def hat(w):
    O = np.zeros(w.shape[:-1] + (3, 3))
    O[..., 0, 1], O[..., 0, 2], O[..., 1, 2] = -w[..., 2], w[..., 1], -w[..., 0]
    return O - np.swapaxes(O, -1, -2)

def W_mat(w, sig):
    """Translation block of the Sim(3) exponential (Strasdat et al., 2010, eq. 22)."""
    th, es = np.linalg.norm(w, axis=-1), np.exp(sig)
    ss = np.abs(sig) < 1e-6
    C = np.where(ss, 1 + sig / 2, np.expm1(sig) / np.where(ss, 1, sig))
    g = th > 1e-5                                   # small angles: limits of the closed form
    a, b, d = es * np.sin(th), es * np.cos(th), sig**2 + th**2
    thg, dg, sg = np.where(g, th, 1), np.where(g, d, 1), np.where(ss, 1, sig)
    A = np.where(g, (a * sig + (1 - b) * th) / (thg * dg),
                 np.where(np.abs(sig) < 1e-4, 1/2 + sig/3 + sig**2/8, (es * (sig - 1) + 1) / sg**2))
    B = np.where(g, (C - ((b - 1) * sig + a * th) / dg) / thg**2,
                 np.where(np.abs(sig) < 1e-4, 1/6 + sig/8 + sig**2/20,
                          (es * (sig**2 - 2*sig + 2) - 2) / (2 * sg**3)))
    K = hat(w)
    return A[..., None, None] * K + B[..., None, None] * (K @ K) + C[..., None, None] * np.eye(3)

def rotvec(w):
    return Rotation.from_rotvec(w.reshape(-1, 3)).as_matrix().reshape(w.shape[:-1] + (3, 3))

def exp7(xi):                                       # xi = (omega, upsilon, sigma)
    w, u, sig = xi[..., :3], xi[..., 3:6], xi[..., 6]
    return np.exp(sig), rotvec(w), np.einsum('...ij,...j->...i', W_mat(w, sig), u)

def log7(S):
    s, R, t = S
    w = Rotation.from_matrix(R.reshape(-1, 3, 3)).as_rotvec().reshape(R.shape[:-2] + (3,))
    sig = np.log(s)
    return np.concatenate([w, np.linalg.solve(W_mat(w, sig), t[..., None])[..., 0], sig[..., None]], -1)

def mul(A, B):
    return A[0] * B[0], A[1] @ B[1], A[0][..., None] * np.einsum('...ij,...j->...i', A[1], B[2]) + A[2]

def inv(A):
    Rt = np.swapaxes(A[1], -1, -2)
    return 1 / A[0], Rt, -np.einsum('...ij,...j->...i', Rt, A[2]) / A[0][..., None]

def take(A, idx):
    return A[0][idx], A[1][idx], A[2][idx]

def centres(S):                                     # camera centres of world-to-camera S_iw
    return -np.einsum('...ji,...j->...i', S[1], S[2]) / S[0][..., None]

# --- ground truth: a closed 110.3 m route, keyframes every metre, 3 m of overlap ---
def route(a, L1=30.0, r=8.0):
    a = a % (2 * L1 + 2 * np.pi * r)
    if a < L1: return np.array([a, 0.0]), 0.0
    a -= L1
    if a < np.pi * r: return np.array([L1 + r * np.sin(a / r), r - r * np.cos(a / r)]), a / r
    a -= np.pi * r
    if a < L1: return np.array([L1 - a, 2 * r]), np.pi
    a -= L1
    return np.array([-r * np.sin(a / r), r + r * np.cos(a / r)]), np.pi + a / r

L_ROUTE = 60 + 16 * np.pi
N = int(L_ROUTE + 3) + 1                            # 114 keyframes; the last is back near keyframe 3
LOOP_KF, CUR_KF = 3, N - 1

def make_truth():
    Rcw, C = [], []
    for k in range(N):
        p, h = route(float(k))
        f = np.array([np.cos(h), np.sin(h), 0.0])
        Rcw.append(np.column_stack([np.cross(f, [0, 0, 1.0]), [0, 0, -1.0], f]).T)  # x right, y down, z ahead
        C.append([p[0], p[1], 1.5])
    Rcw = np.array(Rcw)
    return np.ones(N), Rcw, -np.einsum('kij,kj->ki', Rcw, np.array(C))

def make_landmarks(rng, per_m=30):                  # points 3-6 m to either side, 0-3 m high
    X = []
    for a in rng.uniform(0, L_ROUTE, int(per_m * L_ROUTE)):
        p, h = route(a)
        q = p + rng.choice([-1, 1]) * rng.uniform(3, 6) * np.array([-np.sin(h), np.cos(h)])
        X.append([q[0], q[1], rng.uniform(0, 3)])
    return np.array(X)

# --- monocular estimate: local map scale lam_k drifts along the route ---
def scale_profile(kind, ratio, rng):
    k = np.arange(N) / (N - 1)
    if kind == "steady":
        return np.exp(k * np.log(ratio))
    w = np.concatenate([[0], np.cumsum(rng.normal(0, 0.03, N - 1))])
    return np.exp(k * np.log(ratio) + w - k * w[-1])  # random walk pinned to the same end ratio

def drifted(truth, lam, rng, rot_deg=0.05, trans=0.01):
    rel = mul(take(truth, slice(1, None)), inv(take(truth, slice(None, -1))))   # T_{k+1,k}
    R, t = [truth[1][0]], [truth[2][0]]
    for k in range(N - 1):
        Rr = rotvec(np.radians(rot_deg) * rng.normal(size=3)) @ rel[1][k]
        R.append(Rr @ R[-1])
        t.append(Rr @ t[-1] + lam[k + 1] * (rel[2][k] + trans * rng.normal(size=3)))
    return np.ones(N), np.array(R), np.array(t)

# --- map points, covisibility and the Essential Graph (Mur-Artal et al., 2015, Sec. III-D) ---
def observe(truth, X):
    Xc = np.einsum('kij,nj->kni', truth[1], X) + truth[2][:, None, :]
    z = Xc[..., 2]
    vis = (z > 1) & (z < 15) & (np.abs(Xc[..., 0]) < z) & (np.abs(Xc[..., 1]) < 0.75 * z)
    mp = -np.ones(vis.shape, int)                   # map point instance seen by each keyframe
    last, cur, nid = -100 * np.ones(len(X), int), -np.ones(len(X), int), 0
    for k in range(N):
        for j in np.flatnonzero(vis[k]):
            if k - last[j] > 10:                    # unseen for long: a duplicate map point is created
                cur[j], nid = nid, nid + 1
            mp[k, j], last[j] = cur[j], k
    return Xc, vis, mp

def essential_graph(mp, theta_min=100):
    sets = [set(r[r >= 0]) for r in mp]
    W = np.array([[len(a & b) for b in sets] for a in sets])
    edges = set()
    for k in range(1, N):
        edges.add((int(np.argmax(W[k, :k])), k))    # spanning tree: parent shares most points
        edges |= {(j, k) for j in range(k) if W[k, j] >= theta_min}
    return sorted(edges)

def horn(P1, P2, with_scale):
    """P1 ~ s R P2 + t: Horn's quaternion method, scale as in ORB_SLAM3's Sim3Solver."""
    O1, O2 = P1.mean(0), P2.mean(0)
    Q1, Q2 = P1 - O1, P2 - O2
    (xx, xy, xz), (yx, yy, yz), (zx, zy, zz) = Q2.T @ Q1
    Nq = np.array([[xx + yy + zz, yz - zy, zx - xz, xy - yx],
                   [yz - zy, xx - yy - zz, xy + yx, zx + xz],
                   [zx - xz, xy + yx, -xx + yy - zz, yz + zy],
                   [xy - yx, zx + xz, yz + zy, -xx - yy + zz]])
    q = np.linalg.eigh(Nq)[1][:, -1]
    R = Rotation.from_quat([q[1], q[2], q[3], q[0]]).as_matrix()
    s = np.sum(Q1 * (Q2 @ R.T)) / np.sum((Q2 @ R.T) ** 2) if with_scale else 1.0
    return np.array(s), R, O1 - s * R @ O2

# --- pose graph over Sim(3) (dof 7) or SE(3) (dof 6): identity information, LM, 20 iterations ---
def optimize(S0, edges, Z, fixed, dof, iters=20):
    free = [k for k in range(N) if k != fixed]
    col = {k: c for c, k in enumerate(free)}
    I, J = np.array(edges).T
    Zinv = inv(Z)
    nbr = {k: set() for k in range(N)}
    for i, j in edges:
        nbr[i].add(j); nbr[j].add(i)
    color = {}
    for k in free:                                  # keyframes perturbed together share no residual
        used = {color[m] for q in nbr[k] | {k} for m in nbr[q] if m in color}
        color[k] = min(set(range(len(used) + 1)) - used)
    groups = [[k for k in free if color[k] == c] for c in set(color.values())]
    res = lambda xi: log7(mul(Zinv, mul(take(mul(exp7(xi), S0), J),
                                        inv(take(mul(exp7(xi), S0), I))))).ravel()
    xi, mu = np.zeros((N, 7)), 1e-6
    for _ in range(iters):
        r = res(xi)
        rows, cols, vals = [], [], []
        for g in groups:                            # finite-difference sparse Jacobian
            for d in range(dof):
                x2 = xi.copy(); x2[g, d] += 1e-7
                dr = (res(x2) - r) / 1e-7
                for k in g:
                    rr = (7 * np.flatnonzero((I == k) | (J == k))[:, None] + np.arange(7)).ravel()
                    rows.append(rr); cols.append(np.full(rr.size, dof * col[k] + d)); vals.append(dr[rr])
        Jm = csr_matrix((np.concatenate(vals), (np.concatenate(rows), np.concatenate(cols))),
                        shape=(len(r), dof * len(free)))
        H, grad = (Jm.T @ Jm).tocsc(), Jm.T @ r
        while mu < 1e8:
            step = spsolve(H + mu * diags(H.diagonal()), -grad)
            x2 = xi.copy(); x2[free, :dof] += step.reshape(-1, dof)
            if np.sum(res(x2) ** 2) <= np.sum(r ** 2):
                xi, mu = x2, mu / 10
                break
            mu *= 10
        if mu >= 1e8 or np.abs(step).max() < 1e-12:
            break
    S = mul(exp7(xi), S0)
    return S[0], S[1], S[2]

# --- measures: ATE after Sim(3) alignment of keyframe positions, and the local scale ---
def evaluate(S, truth):
    P, Q = centres(S), centres(truth)
    mp, mq = P.mean(0), Q.mean(0)
    U, D, Vt = np.linalg.svd((Q - mq).T @ (P - mp) / N)
    E = np.diag([1, 1, np.sign(np.linalg.det(U @ Vt))])
    s = np.trace(np.diag(D) @ E) / np.mean(np.sum((P - mp) ** 2, 1))       # Umeyama
    ate = np.sqrt(np.mean(np.sum((s * (P - mp) @ (U @ E @ Vt).T - (Q - mq)) ** 2, 1)))
    step = s * np.linalg.norm(np.diff(P, axis=0), axis=1) / np.linalg.norm(np.diff(Q, axis=0), axis=1)
    return ate, step.min(), step.max()

def run(kind, ratio, seed=0, fixed=LOOP_KF):
    rng = np.random.default_rng(seed)
    truth = make_truth()
    X = make_landmarks(rng)
    Xc, vis, mp = observe(truth, X)
    edges = essential_graph(mp)
    lam = scale_profile(kind, ratio, rng)
    est = drifted(truth, lam, rng)
    Z = mul(take(est, [j for i, j in edges]), inv(take(est, [i for i, j in edges])))  # scale 1
    both = vis[LOOP_KF] & vis[CUR_KF]               # the same landmarks, in two map-point copies
    noisy = lambda k: lam[k] * (Xc[k, both] + 0.01 * Xc[k, both, 2:] * rng.normal(size=(both.sum(), 3)))
    P_l, P_i = noisy(LOOP_KF), noisy(CUR_KF)
    out = {"none": evaluate(est, truth)}
    for name, dof in [("SE(3)", 6), ("Sim(3)", 7)]:
        S_il = horn(P_i, P_l, with_scale=(dof == 7))
        Zall = tuple(np.concatenate([z, np.asarray(m)[None]]) for z, m in zip(Z, S_il))
        out[name] = evaluate(optimize(est, edges + [(LOOP_KF, CUR_KF)], Zall, fixed, dof), truth)
        out["loop scale"] = float(S_il[0])
    return out, lam[CUR_KF] / lam[LOOP_KF]

print("profile      end  true/loop scale | ATE (m) and local scale range")
print("                  ratio  measured | none                  SE(3)                 Sim(3)")
for kind in ["steady", "random walk"]:
    for ratio in [1.0, 0.7, 0.4]:
        o, true_ratio = run(kind, ratio)
        cols = "  ".join(f"{o[m][0]:6.3f} {o[m][1]:4.2f}-{o[m][2]:4.2f}" for m in ["none", "SE(3)", "Sim(3)"])
        print(f"{kind:11s} {ratio:4.1f}  {true_ratio:5.3f}  {o['loop scale']:6.3f}  | {cols}")
o, _ = run("steady", 0.4, fixed=0)
print(f"steady 0.4 with the first keyframe fixed instead: Sim(3) ATE {o['Sim(3)'][0]:.3f} m")
for kind in ["steady", "random walk"]:
    a = np.array([[run(kind, 0.4, seed)[0][m][0] for m in ["none", "SE(3)", "Sim(3)"]] for seed in range(5)])
    print(f"{kind}, end ratio 0.4, seeds 0-4: ATE none {a[:, 0].min():.2f}-{a[:, 0].max():.2f} m, "
          f"SE(3) {a[:, 1].min():.2f}-{a[:, 1].max():.2f} m, Sim(3) {a[:, 2].min():.3f}-{a[:, 2].max():.3f} m")

Output:

profile      end  true/loop scale | ATE (m) and local scale range
                  ratio  measured | none                  SE(3)                 Sim(3)
steady       1.0  1.000   1.000  |  0.058 0.98-1.03   0.051 0.98-1.03   0.051 0.98-1.03
steady       0.7  0.707   0.707  |  1.625 0.84-1.21   1.395 0.84-1.22   0.275 0.91-1.07
steady       0.4  0.410   0.410  |  4.304 0.61-1.53   3.614 0.61-1.57   0.658 0.81-1.16
random walk  1.0  1.015   1.016  |  0.937 0.86-1.18   0.685 0.86-1.18   0.515 0.88-1.14
random walk  0.7  0.718   0.718  |  1.092 0.87-1.27   1.053 0.87-1.26   0.425 0.89-1.13
random walk  0.4  0.416   0.416  |  3.663 0.65-1.62   3.293 0.64-1.63   0.596 0.82-1.22
steady 0.4 with the first keyframe fixed instead: Sim(3) ATE 0.658 m
steady, end ratio 0.4, seeds 0-4: ATE none 4.25-4.30 m, SE(3) 3.59-3.62 m, Sim(3) 0.645-0.708 m
random walk, end ratio 0.4, seeds 0-4: ATE none 3.52-5.16 m, SE(3) 3.09-4.03 m, Sim(3) 0.495-0.983 m

Without drift, the three estimates agree to within a centimetre: the loop measurement reports a scale change of 1.000, both corrections close the small remaining gap the same way, and the local scale stays within the 2–3% spread that the per-step noise produces. This is the base case that both corrections must reproduce, and both do.

In every case, Horn’s method recovers the true scale ratio between the two ends of the loop to within about 0.001, so the difference between the corrections lies in what the graph can do with that measurement. The SE(3)SE(3) correction can only bend and shift the chain to close the loop. In the four cases with an end ratio below 1, it lowers the ATE by only 4–16%, and its local-scale range is about the same as before correction: the parts of the route mapped at 0.6 or 1.5 times the true scale remain so. The Sim(3)Sim(3) correction lets every keyframe change scale, and the edges, which all carry scale 1, spread the measured scale change around the loop. In those cases it lowers the ATE about sixfold, or by a factor of 2.6 for the random walk to 0.7, and narrows the local scale from 0.61–1.53 to 0.81–1.16 at the steady end ratio of 0.4. This is the comparison Strasdat et al. (2010, §V) drew qualitatively on a real sequence (Fig. 2) and by trajectory RMSE in simulation (Fig. 3), here in ORB-SLAM’s graph.

The correction is not complete, even for steady drift, where the relative scale between consecutive keyframes is the same along the whole route. In our reading, two features of the set-up contribute: with identity information matrices, the single loop edge weighs no more than any one of the hundreds of tree and covisibility edges, each of which is violated when the scale changes between its two keyframes, so the least-squares solution leaves part of the measured scale change on the loop edge; and the example leaves out the extra edges that ORB-SLAM adds between the neighbours of the two loop keyframes. The random walk shows a limit of the method itself: the graph’s edges all say “scale 1”, so nothing in it records where along the route the scale changed, and the optimizer distributes the correction without regard to where the drift happened. With the walk pinned to an end ratio of 1.0, the loop measures a scale change of under 2%, and neither correction can remove the 0.86–1.18 spread of local scale; Sim(3)Sim(3) still lowers the ATE more, from 0.94 to 0.52 m. Over seeds 0–4 at an end ratio of 0.4, the Sim(3)Sim(3) ATE varies little for steady drift and about twofold for the random walk, while the lowest SE(3)SE(3) ATE is more than three times the highest Sim(3)Sim(3) ATE. Fixing the first keyframe instead of the loop keyframe, as the ORB-SLAM3 code does, gives the same ATE: the residuals depend only on relative transformations, so the fixed keyframe chooses the gauge without changing the aligned trajectory.

The example runs in about 70 seconds with NumPy 2.3.5 and SciPy 1.16.3 and gives identical output on repeated runs. It isolates the pose-graph step: ORB-SLAM follows it, from ORB-SLAM2 on, with full bundle adjustment over the points, which can also use the observations that the graph discards.

Properties and Behavior

Accuracy on TUM RGB-D. On 16 hand-held sequences of TUM RGB-D, using the colour images only, Mur-Artal et al. (2015, Table III) compared keyframe trajectories after a seven-degree-of-freedom alignment, with the median over five runs. ORB-SLAM’s keyframe RMSE ranged from 0.30 cm (fr2_xyz) to 3.81 cm (fr2_360_kidnap). It processed every sequence except fr3_nostructure_texture_far, a planar scene with the twofold ambiguity, where its initialization detected the ambiguity and refused to start. PTAM lost track in eight sequences and LSD-SLAM in three. ORB-SLAM and PTAM had similar accuracy on open trajectories, ORB-SLAM was more accurate on sequences with large loops, and both were clearly more accurate than LSD-SLAM and RGBD-SLAM. The authors suggest the cause is that those methods reduce map optimization to a pose graph, discarding sensor measurements, while bundle adjustment optimizes over them (§VIII-B, §IX-B); the trade-offs are discussed with direct methods.

Relocalization. After building a map from the first 30 s of fr2_xyz, ORB-SLAM relocalized 78.4% of the remaining 2,769 frames with an RMSE of 0.38 cm, against 34.9% for PTAM. From a map of fr3_sitting_xyz it relocalized 77.9% of the 859 frames of fr3_walking_xyz, with people moving through the scene, where PTAM relocalized none (2015, Table IV).

Lifelong operation. Looking at one desk for 93 s from changing viewpoints, PTAM kept inserting keyframes while ORB-SLAM’s number of keyframes levelled off. Over six consecutive dynamic sequences of the same desk, the map grew while new views appeared, stopped growing when the map explained the scene, and grew again with new content that the authors attribute probably to dynamic changes; most keyframes were culled soon after creation (2015, §VIII-D, Figs. 9–10).

Large scale. On the KITTI odometry sequences, run at their 10 fps recording rate, ORB-SLAM processed all sequences except 01, a highway with few trackable close objects, and closed the loops in 00, 02, 05, 06, 07 and 09; the loop in 09 appears only in a few frames at the end, and the reported results are for the runs in which it was detected. The keyframe RMSE after similarity alignment was typically about 1% of the map’s dimensions, from 1.59 m on sequence 03 (471 × 199 m) to 46.58 m on 08 (808 × 391 m), which has no loops to correct its drift, especially in scale; 20 iterations of full bundle adjustment at the end slightly improved trajectories with loops and made little difference to open ones (2015, Table V, Fig. 12).

Non-determinism. The threads run alongside other processes, so results vary from run to run, which is why the authors report medians over several runs (2015, §VIII). Campos Martínez (2021, §4.4.1) attributes ORB-SLAM3’s run-to-run variation to RANSAC and thread scheduling and reports that inertial measurements reduce it.

Limitations

  • Monocular scale. The map has no metric scale, and between loop closures its scale drifts; on an open trajectory nothing corrects it (2015, Fig. 12, sequence 08). ORB-SLAM2 and ORB-SLAM3 address this with stereo, depth or inertial sensing.
  • Initialization needs parallax. The heuristic delays the start until a two-view configuration is safe, and refuses ambiguous planar configurations altogether (2015, §IV, §VIII-B).
  • Points without parallax. Points too far to be triangulated are left out of the map, although they carry rotation information; the authors list them as future work (2015, §IX-C). ORB-SLAM2’s far stereo points use some of this information (below).
  • Texture. Everything depends on ORB features, and Campos Martínez (2021, §4.5) names low-texture environments as ORB-SLAM3’s main failure case. On KITTI 01, a highway with few trackable close objects, the monocular system could not process the sequence (2015, §VIII-E).
  • Place recognition. Loops traversed in opposite directions were not detected on NewCollege, so the big loop on the right of the map stayed misaligned (2015, Fig. 6). The requirement of three consecutive consistent detections also delays loop closures and can miss them (Elvira López-Echazarreta, 2024, §4.2.1); ORB-SLAM3 changed this (below).
  • Sparse map. The map is a set of points for localization, not a surface; the authors point to keyframes as a basis for denser reconstruction (2015, §IX-C).
  • Static world. The system has no model of moving objects; the paper shows it working under what the authors call moderate dynamic changes, such as the people moving in the TUM RGB-D relocalization and lifelong experiments (2015, §VIII-C–D).

Variants

ORB-SLAM2: stereo and RGB-D

ORB-SLAM2 keeps the three threads and adds stereo and RGB-D input, a full bundle adjustment after loop closure and a localization mode (Mur Artal, 2017, Ch. 6, which presents the system described in the 2017 paper, then under review). The input is reduced to keypoints early, so the rest of the system does not depend on the sensor:

  • Stereo keypoints (uL,vL,uR)(u_L, v_L, u_R) come from matching ORB features along the rows of rectified images, with the right coordinate refined to sub-pixel accuracy. For RGB-D frames a virtual right coordinate is synthesized from the depth dd as uR=uL−fxb/du_R = u_L - f_x b/d, with the baseline bb between the projector and the infrared camera approximated as 8 cm for Kinect and Asus Xtion cameras.
  • Close and far points. A stereo keypoint is close if its depth is below 40 times the baseline and far otherwise. Close points are triangulated from a single frame and constrain scale, translation and rotation; far points mainly constrain rotation and are triangulated only when several views support them; features without a stereo match are used as monocular observations. A keyframe is also inserted when the number of tracked close points drops below 100 and the frame could create at least 70 new close points.
  • Rigid loop closing. With scale observable, loop verification and the pose graph use rigid transformations instead of similarities. A full bundle adjustment then runs in a separate thread; if a new loop is detected meanwhile it is aborted and relaunched, and its result is merged with keyframes inserted during the optimization by propagating the corrections through the spanning tree.
  • Localization mode turns off local mapping and loop closing and tracks against the map, using matches to map points and visual odometry matches to points created from the previous frame’s stereo or depth data.

Run five times per sequence, with the median reported, ORB-SLAM2 had a relative translation error below 1% on most stereo KITTI sequences, from 0.40% to 1.39% with the highway sequence 01 the worst, and RMS translation errors of 1.8 to 11.9 cm on the EuRoC MAV sequences it completed, losing track in parts of V2_03 (Mur Artal, 2017, Tables 6.1–6.2). On seven RGB-D sequences of TUM RGB-D it was the only method in the comparison based on bundle adjustment and the most accurate on most sequences; the runs compensated a 4% scale bias in the fr2 depth maps, which the author notes could partly explain the better results (§6.2.3, Table 6.3).

Mur-Artal and Tardós also built a monocular visual-inertial ORB-SLAM with map reuse and an IMU initialization that estimates scale, gravity, velocity and biases (Mur Artal, 2017, Ch. 7), on which ORB-SLAM3 builds.

ORB-SLAM3: visual-inertial and multi-map

ORB-SLAM3 runs in pure visual or visual-inertial mode with monocular, stereo or RGB-D cameras, and with pinhole or fisheye lenses through an abstract camera model that supplies projection, unprojection and Jacobians (Campos Martínez, 2021, Ch. 4; Elvira López-Echazarreta, 2024, Ch. 4, whose §4.1 is joint work with the paper’s other authors). That joint description lists four novelties: place recognition with improved recall, the Atlas of maps, inertial integration with a new initialization, and the abstract camera model (Elvira López-Echazarreta, 2024, §4.1). The first three, with map merging, work as follows:

  • Visual-inertial SLAM based throughout on maximum a posteriori estimation. The IMU is initialized by an inertial-only optimization of scale, gravity direction, velocities and biases with the visual keyframe poses fixed, followed by joint visual-inertial bundle adjustment; for stereo the scale is fixed to one. A scale and gravity refinement then runs in local mapping every ten seconds until the map has more than 100 keyframes or 75 s have passed. Tracking optimizes only the states of the last two frames, with map points fixed and the older frame marginalized into a prior, and local mapping optimizes a window of 10 keyframes, doubled when tracking is strong and keyframes arrive slowly (Campos Martínez, 2021, §4.3). Camera–IMU fusion in general is the subject of visual-inertial odometry.
  • Atlas, a set of disconnected maps. Tracking works in an active map; when it is lost, it tries to relocalize in every map of the Atlas, and if that fails for long enough, it stores the active map and starts a new one. A single DBoW2 database covers all maps and serves relocalization, loop closing and map merging (Elvira López-Echazarreta, 2024, §4.1).
  • Place recognition with higher recall. DBoW2’s temporal consistency check needs three keyframes before geometric verification, which the authors found too slow, leaving duplicated areas. ORB-SLAM3 retrieves the three most similar keyframes for each new keyframe, builds a local window around each with its best covisible keyframes, estimates the alignment from 3D–3D matches with Horn’s method in RANSAC, Sim(3)Sim(3) for pure monocular and immature monocular-inertial maps and SE(3)SE(3) otherwise, refines it by guided matching and a bidirectional reprojection error, and then checks it in covisible keyframes of the query that are usually already in the map. In visual-inertial mode, pitch and roll must also agree, because gravity makes them observable (Elvira López-Echazarreta, 2024, §4.2.1).
  • Merging. A match in another map triggers a merge: the keyframes and points around the two matched keyframes (the “welding window”) are aligned, duplicate points are fused, a local bundle adjustment over the welding window refines the junction, and an Essential Graph optimization carries the correction to the rest of the merged map. A match in the active map is closed the same way, followed by a full bundle adjustment (§4.2.2–4.2.4).

In visual-inertial mode, scale, roll and pitch are observable, so the pose graph after a loop updates only four degrees of freedom (Campos Martínez, 2021, §4.3.5). The two theses differ on what follows: Campos Martínez writes that the full bundle adjustment can be skipped in the visual-inertial case, while Elvira López-Echazarreta writes that it runs only when the map has fewer keyframes than a threshold. The released code launches it when the IMU is not yet initialized, or when the map has fewer than 200 keyframes and the Atlas holds a single map (LoopClosing.cc).

Both theses evaluate on the eleven EuRoC MAV sequences with RMS ATE after Sim(3)Sim(3) alignment for pure monocular and SE(3)SE(3) alignment otherwise, scale error ∣1−s∣|1 - s| from the Sim(3)Sim(3) alignment, and the median of ten runs. The table in Elvira López-Echazarreta (2024, Table 4.1), which reproduces the paper’s results, gives an average of 3.5 cm for stereo-inertial ORB-SLAM3 against 13.8 cm for stereo-inertial VINS-Fusion, and 4.3 cm for monocular-inertial ORB-SLAM3 against 11.0 cm for VINS-Mono, both run by the authors with default settings; Campos Martínez (2021, Table 4.1 and text) gives 3.6 cm for stereo-inertial, with slightly different per-sequence values. On the difficult monocular sequences V1_03 and V2_03, the multi-map system covered 94% and 88% of the trajectory with two maps, against 10% and 16% for single-map ORB-SLAM (Elvira López-Echazarreta, 2024, Table 4.2). The protocols are described with EuRoC MAV.

Direct Sparse Odometry, which direct methods compares with ORB-SLAM, minimizes photometric rather than reprojection errors. Visual odometry covers systems that keep only a local window, and pose graph optimization and loop closure cover the back-end and place-recognition problems in general.

Related

  • Bundle Adjustment

    The joint nonlinear least-squares refinement of all camera parameters and 3D points in a reconstruction, made practical by the sparse structure of the problem and the Schur complement.

  • Visual Odometry

    Estimating the motion of a camera, and of the vehicle or device carrying it, incrementally from a video stream, frame after frame, without building a globally consistent map.

  • Bag of Visual Words

    Representing an image by counts of quantized local descriptors ("visual words") from a learned vocabulary; how vocabularies are built, how descriptors are assigned to words, how tf-idf weighting and inverted files make retrieval fast, and the vocabulary tree, approximate k-means and soft assignment that scaled it to millions of images.

  • Visual Place Recognition

    Recognizing where a photograph was taken by retrieving geotagged images of the same place; the retrieval formulation, FAB-MAP, aggregated descriptors (VLAD, NetVLAD, GeM), geometric re-ranking, appearance and viewpoint change, its role in SLAM loop closure, foundation-model features, and the Pittsburgh, Tokyo 24/7, Nordland and Mapillary SLS benchmarks with Recall@N and the localization radius.

  • 2D and 3D Geometric Transformations

    The nested families of Euclidean, similarity, affine and projective transformations of the plane and of space, their degrees of freedom, what each preserves, why each forms a group, and the Lie groups SO(3), SE(3) and Sim(3) used to estimate rotations, poses and similarities.

  • Absolute Trajectory Error

    The root-mean-square distance between estimated and ground-truth camera positions after the two trajectories are aligned; how the alignment (SE(3), Sim(3), yaw-only, all poses or the first), timestamp association and reference frames change the number, and how it relates to the relative pose error and the KITTI odometry metric.

References

  1. Mur-Artal, R., Montiel, J. M. M. & Tardós, J. D. (2015). ORB-SLAM: A Versatile and Accurate Monocular SLAM System. IEEE Transactions on Robotics, 31(5), 1147–1163.
  2. Mur-Artal, R. & Tardós, J. D. (2017). ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras. IEEE Transactions on Robotics, 33(5), 1255–1262 (not read).
  3. Campos, C., Elvira, R., Gómez Rodríguez, J. J., Montiel, J. M. M. & Tardós, J. D. (2021). ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual–Inertial, and Multimap SLAM. IEEE Transactions on Robotics, 37(6), 1874–1890 (not read).
  4. Mur Artal, R. (2017). Real-Time Accurate Visual SLAM with Place Recognition. PhD thesis, Universidad de Zaragoza.
  5. Campos Martínez, C. (2021). Precise and Robust Visual SLAM with Inertial Sensors and Deep Learning. PhD thesis, Universidad de Zaragoza.
  6. Elvira López-Echazarreta, R. (2024). Robust Discrete Feature-Based Multiple-Map for Visual SLAM in Real-World and Endoscopy Applications. PhD thesis, Universidad de Zaragoza.
  7. Gálvez-López, D. & Tardós, J. D. (2012). Bags of Binary Words for Fast Place Recognition in Image Sequences. IEEE Transactions on Robotics, 28(5), 1188–1197.
  8. Strasdat, H., Montiel, J. M. M. & Davison, A. J. (2010). Scale Drift-Aware Large Scale Monocular SLAM. Robotics: Science and Systems (RSS) VI.
  9. Mur-Artal, R. & Tardós, J. D. ORB-SLAM2 source code (commit f2e6f51). GitHub repository raulmur/ORB_SLAM2.
  10. Campos, C., Elvira, R., Gómez Rodríguez, J. J., Montiel, J. M. M. & Tardós, J. D. ORB-SLAM3 source code (V1.0, commit 4452a3c). GitHub repository UZ-SLAMLab/ORB_SLAM3.