DUSt3R

DUSt3R (“Dense and Unconstrained Stereo 3D Reconstruction”) is a network for 3D reconstruction from images whose cameras are unknown: neither their intrinsic parameters nor their poses are given. For a pair of images it regresses a pointmap for each, a 3D point for every pixel, with both pointmaps expressed in the coordinate frame of the first camera, together with a confidence for every pixel. Depth, focal length, relative pose and pixel correspondences are then read off the two pointmaps, and for more than two images a global alignment brings the pairwise predictions into one frame. It was introduced by Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii and Jerome Revaud of Aalto University and NAVER LABS Europe at CVPR 2024. The network is a transformer: a vision transformer encoder shared by the two images and two decoders that exchange information through cross-attention, initialized from CroCo, a cross-view completion pretraining of the same group. MASt3R (“Matching And Stereo 3D Reconstruction”; Leroy, Cabon and Revaud, ECCV 2024) adds a head of dense local features trained for matching, predicts metric pointmaps where the training data allow, and comes with a fast reciprocal matcher.

This article reads both papers and their supplements in the official open-access copies (CVF for DUSt3R, ECVA for MASt3R); section, equation and table numbers are from those copies. Code details come from the official repositories at pinned commits (references). Each detail below names the model it belongs to.

Problem Addressed

Dense reconstruction from photographs is usually a chain of separate steps: keypoint detection and feature matching, robust two-view estimation, structure from motion with bundle adjustment to recover the cameras, and then multi-view stereo, which needs those cameras to triangulate. The DUSt3R paper argues that each step adds noise to the next, that the steps do not inform each other (the sparse reconstruction that gives the cameras and the dense one built on them could help each other but do not), and that SfM, the step that estimates every camera parameter, fails in common situations: few views, non-Lambertian surfaces, too little or too much camera motion. Earlier learned two-view and multi-view networks such as DeMoN and DeepV2D also needed the ground-truth intrinsics as input and output a depth map and a relative pose.

DUSt3R removes the camera model from the network’s output. A pointmap from the pair, both images’ pixels as 3D points in one frame, contains the scene geometry, the pixel-to-point relation and the relation between the two viewpoints at once; the paper’s claim is that nearly every camera and scene parameter can then be recovered from it, and that the same network covers the monocular case when it is given the same image twice.

Architecture

A pointmap $X \in \mathbb{R}^{W \times H \times 3}$ assigns a 3D point to every pixel of an image $I$ of size $W \times H$ , assuming that each camera ray hits a single surface (no translucency). With intrinsics $K$ and a depth map $D$ , the pointmap in the camera’s own frame is $X_{i,j} = K^{-1} D_{i,j} [i, j, 1]^\top$ , the back-projection of the pinhole camera model. DUSt3R writes $X^{n,m}$ for the pointmap of image $n$ expressed in camera $m$ ’s frame (Eq. 1):

X^{n,m} = P_m P_n^{-1} h\!\left(X^{n}\right),

with $P_m, P_n \in \mathbb{R}^{3 \times 4}$ the world-to-camera poses and $h$ the map to homogeneous coordinates. The network $f$ takes two images $I^1, I^2$ and outputs $X^{1,1}$ and $X^{2,1}$ , both in the first camera’s frame, with confidence maps $C^{1,1}, C^{2,1} \in \mathbb{R}^{W \times H}$ . The output is therefore asymmetric: camera 1 sits at the origin, and camera 2’s position is implicit in where the second image’s points land.

Encoder

Both images are split into $16 \times 16$ patches and encoded in a Siamese manner by the same ViT encoder, giving token sets $F^1 = \operatorname{Encoder}(I^1)$ and $F^2 = \operatorname{Encoder}(I^2)$ . In the released models the encoder is a ViT-Large (§4). The training commands in the README use rotary position embeddings (pos_embed='RoPE100'), for which the README offers CUDA kernels “as in CroCo v2”.

Decoder

Two transformer decoders, one per branch, process the tokens jointly. Each decoder block performs self-attention over its own view’s tokens, then cross-attention from its view’s tokens to all tokens of the other view, then an MLP, so that information passes between the branches at every block:

G_i^1 = \operatorname{DecoderBlock}_i^1\!\left(G_{i-1}^1, G_{i-1}^2\right), \qquad G_i^2 = \operatorname{DecoderBlock}_i^2\!\left(G_{i-1}^2, G_{i-1}^1\right),

for $i = 1, \dots, B$ , starting from $G_0^1 = F^1$ and $G_0^2 = F^2$ . The paper calls this exchange crucial for producing aligned pointmaps. The decoders are ViT-Base sized (12 blocks of width 768 in the README’s model string).

Regression Heads

A separate head per branch maps the decoder tokens of all blocks to a pointmap and a confidence map, $X^{1,1}, C^{1,1} = \operatorname{Head}^1(G_0^1, \dots, G_B^1)$ and likewise for the second branch. The released models use either a linear head or a DPT head, the dense prediction transformer decoder also used in MiDaS v3. In the code (heads/postprocess.py) the raw 3D output is mapped by depth_mode=('exp', -inf, inf), which keeps the direction of the predicted vector and replaces its length $d$ by $e^{d} - 1$ , and the confidence by conf_mode=('exp', 1, inf), that is $C = 1 + e^{c}$ , the strictly positive form the paper gives.

The architecture enforces no geometric constraint: the pointmaps need not correspond to any pinhole camera, and they are predicted only up to an unknown scale. The paper reports that in practice they closely fit one (supplementary §E, below).

Training

Loss. DUSt3R is trained by regression in 3D only. For a valid pixel $i$ of view $v \in \{1, 2\}$ , with ground-truth pointmaps $\bar X^{1,1}$ and $\bar X^{2,1}$ computed from known depth, intrinsics and poses (supplementary §H.1), the regression loss is the Euclidean distance after normalizing each pair (Eqs. 2–3):

\ell_{\text{regr}}(v, i) = \left\lVert \frac{1}{z} X_i^{v,1} - \frac{1}{\bar z} \bar X_i^{v,1} \right\rVert, \qquad \operatorname{norm}(X^1, X^2) = \frac{1}{|\mathcal{D}^1| + |\mathcal{D}^2|} \sum_{v \in \{1,2\}} \sum_{i \in \mathcal{D}^v} \lVert X_i^{v} \rVert,

where $z = \operatorname{norm}(X^{1,1}, X^{2,1})$ and $\bar z = \operatorname{norm}(\bar X^{1,1}, \bar X^{2,1})$ are the mean distances of all valid points to the origin and $\mathcal{D}^v$ is the set of pixels with ground truth. The confidence enters through the confidence-aware loss (Eq. 4):

\mathcal{L}_{\text{conf}} = \sum_{v \in \{1,2\}} \sum_{i \in \mathcal{D}^v} C_i^{v,1}\, \ell_{\text{regr}}(v, i) - \alpha \log C_i^{v,1},

with $\alpha = 0.2$ (supplementary §H.2). The paper cites Kendall, Gal and Cipolla’s uncertainty weighting for the regularization term and notes that the confidence is learned without any explicit supervision; it motivates the confidence by ill-defined points, such as sky or translucent objects, and by parts of the image that are harder to predict, and the lower bound $C > 1$ forces the network to extrapolate in hard areas, such as those seen by one view only. The official ConfLoss averages the per-pixel terms of each view rather than summing them.

It follows from Eq. 4, by our derivation, what the learned confidence should approach. For a pixel with expected error $\ell$ , the term $C\ell - \alpha \log C$ is minimized at $C^\ast = \alpha / \ell$ , where it equals $\alpha \log \ell$ plus a constant; with the constraint $C \ge 1$ the minimum is at $C = 1$ whenever $\ell \ge \alpha$ , where the term is $\ell$ itself. Accurate pixels thus pay $\alpha$ times the logarithm of their error, plus a constant, and get a confidence inversely proportional to it, while pixels whose normalized error exceeds $\alpha$ get the lowest confidence and pay their error in full. The example below uses $C^\ast$ to simulate confidence maps.

Data. DUSt3R is trained on a mixture of eight datasets with depth or 3D ground truth: Habitat, MegaDepth, ARKitScenes, Static Scenes 3D, BlendedMVS, ScanNet++, CO3D-v2 and Waymo. They cover synthetic and real, indoor, outdoor and object-centric scenes. Where a dataset provides no pairs, the pairs are mined with off-the-shelf image retrieval and point matching, giving 8.5 million pairs in total (§4; supplementary Table 6 lists, for example, 2.04 million from ARKitScenes and 1.76 million from MegaDepth). The supplement’s Table 6 names the synthetic object set “Static Thing 3D” where the main text says “Static Scenes 3D”, and the README lists it as StaticThings3D. The README also notes that the pair lists it publishes are not strictly the ones used for training.

Pretraining. The network is initialized from a CroCo model. CroCo (Weinzaepfel et al., 2022) pretrains a Siamese ViT encoder and a cross-attention decoder on pairs of views of the same scene: most patches of the first image are masked, and the decoder reconstructs them from the visible ones and the second image, a pretext task that rewards understanding the spatial relation between the two views; its pairs were rendered with the Habitat simulator, and it found high masking ratios, such as 90%, to work best. DUSt3R’s main text cites the CroCo paper for the initial weights, while its supplementary Table 5 gives the initialization as CroCo v2 and the README’s training commands start from a CroCo_V2_ViTLarge_BaseDecoder checkpoint; CroCo v2 (Weinzaepfel et al., ICCV 2023) was not read for this article.

Procedure. Training runs in three stages (supplementary Table 5): a linear head at $224 \times 224$ for 50 epochs of 700,000 pairs, then a linear head at 512 pixels in the largest dimension with aspect ratios from $512 \times 384$ to $512 \times 160$ for 100 epochs of 70,000 pairs, then a DPT head at the same resolutions for 90 epochs. All stages use AdamW with base learning rate $10^{-4}$ , weight decay 0.05 and cosine decay. Each epoch samples an equal number of pairs from every dataset. Augmentation is colour jitter and random crops centred on the image, which the supplement describes as a focal-length augmentation: datasets captured with one or a few cameras would otherwise show the network few focal lengths, and centring keeps the principal point in the middle. Each pair is also fed in the reverse order, $(I^2, I^1)$ . The README’s “Our Hyperparameters” commands differ from Table 5 in places: the 224-pixel stage runs 100 epochs, and the mixture has an unreleased internal dataset and no Waymo.

Inference

Reading Off Geometry from One Pair

The paper’s §3.3 lists what one pass of $f(I^1, I^2)$ gives, and how.

Depth. The depth map of image 1 is the third coordinate of $X^{1,1}$ (supplementary Eq. 3), up to the pair’s unknown scale. For monocular depth the same image is fed twice, $f(I, I)$ .
Focal length. Assuming square pixels and a principal point at the image centre, the focal length of camera 1 minimizes the confidence-weighted distance between each pixel’s centred coordinates and the projection of its predicted point:

f_1^\ast = \arg\min_{f_1} \sum_{i=0}^{W} \sum_{j=0}^{H} C_{i,j}^{1,1} \left\lVert (i', j') - f_1 \frac{\left(X_{i,j,0}^{1,1}, X_{i,j,1}^{1,1}\right)}{X_{i,j,2}^{1,1}} \right\rVert, \qquad i' = i - \tfrac{W}{2},\; j' = j - \tfrac{H}{2},

solved in a few iterations of the Weiszfeld algorithm, the iteratively reweighted least-squares method for sums of unsquared distances. The ratio $x/z$ does not change when the pointmap is scaled, so the unknown scale does not matter. Camera 2’s focal length comes from the reversed pass, using $X^{2,2}$ . The global-alignment initialization in the code calls estimate_focal_knowing_depth in its 'weiszfeld' mode, which starts from the closed-form least-squares focal and runs ten reweighting steps without the confidence weights; the function’s default mode, 'median', takes the median of per-pixel focal votes.

Correspondences. Pixels of the two images match when their 3D points are mutual nearest neighbours in the shared frame: $(a, b)$ is kept if $b$ is the nearest point to $a$ among image 2’s points and $a$ the nearest to $b$ among image 1’s.
Relative pose. One route is to match pixels as above, estimate the intrinsics, and recover the pose from the essential matrix. The more direct route compares the same image’s pointmap in two frames, $X^{1,1}$ from $f(I^1, I^2)$ and $X^{1,2}$ from $f(I^2, I^1)$ , by a weighted Procrustes alignment that also gives the relative scale $\sigma^\ast$ :

P^\ast = \sigma^\ast [R^\ast \mid \mathbf{t}^\ast] = \arg\min_{\sigma, R, \mathbf{t}} \sum_i C_i^{1,1} C_i^{1,2} \left\lVert \sigma (R X_i^{1,1} + \mathbf{t}) - X_i^{1,2} \right\rVert^2,

in closed form (the Kabsch algorithm with Umeyama’s scale). Because the two pointmaps describe the same pixels, the correspondences are given. The paper notes that Procrustes is sensitive to noise and outliers and that RANSAC with PnP is more robust. It names $X^{2,2} \leftrightarrow X^{1,2}$ as an equivalent pair; in our reading that pair mixes two images, and the equivalent comparison is $X^{2,1} \leftrightarrow X^{2,2}$ , image 2’s points in the two frames, which the example below uses.

Absolute pose. For visual localization against a reference image with known 2D–3D correspondences, the query’s intrinsics come from its own pointmap, and the pose either from PnP-RANSAC on pixel matches with the reference or from the relative pose, rescaled by comparing the reference’s predicted pointmap with its ground truth.

Global Alignment

For a set of images, DUSt3R builds a graph whose edges are pairs that share content, found either by image retrieval or by passing all pairs through the network, which the paper times at about 25 ms per pair on an H100 GPU, and dropping pairs of low average confidence. Each edge $e = (n, m)$ gives pointmaps $X^{n,e} := X^{n,n}$ and $X^{m,e} := X^{m,n}$ , both in camera $n$ ’s frame and at the pair’s own scale. Global alignment introduces a rigid pose $P_e$ and a scale $\sigma_e > 0$ per edge and solves for world-frame pointmaps $\chi^n$ (Eq. 5):

\chi^\ast = \arg\min_{\chi, P, \sigma} \sum_{e \in \mathcal{E}} \sum_{v \in e} \sum_{i=1}^{HW} C_i^{v,e} \left\lVert \chi_i^{v} - \sigma_e P_e X_i^{v,e} \right\rVert,

with $\prod_e \sigma_e = 1$ to exclude the trivial solution $\sigma_e = 0$ . Since both pointmaps of an edge are in the same frame, one transform per edge must carry both onto the world. To recover cameras as well, $\chi^n$ is replaced by a pinhole parametrization, $\chi_{i,j}^n = P_n^{-1} h\!\left(K_n^{-1} D_{i,j}^n [i, j, 1]^\top\right)$ , so that the unknowns become each image’s pose $P_n$ , intrinsics $K_n$ and depth map $D^n$ . The supplement states that intrinsics are optimized independently for every image, even for frames from one camera.

Unlike bundle adjustment, the objective measures distances in 3D, not reprojection errors in the image. The paper minimizes it by gradient descent, which it says typically converges after a few hundred steps, in seconds on a GPU. The supplement (§G) starts the learning rate between 0.01 and 0.05 and decays it linearly to zero over a few hundred iterations, and it initializes every parameter heuristically: focal lengths by the read-out above; a maximum spanning tree of the graph, each edge scored by the sum of its two mean confidences; pairwise Procrustes alignments propagated along the tree to give a first $\chi^n$ ; Procrustes again for the edges off the tree; RANSAC-PnP for the image poses; and depth maps from the posed $\chi^n$ .

The official implementation (cloud_opt) differs in a few defaults from the equations, as of the pinned commit. It weights the distances by $\log C$ rather than $C$ (conf='log'), so that pixels with confidence near 1 get almost no weight; it averages the distances over each edge’s pixels and then over edges; it fixes the mean log pair scale at $\log 0.5$ instead of the product at 1; it parametrizes depth and focal length in log space and poses by a unit quaternion and a log-compressed translation; and it uses Adam. The README’s usage example runs 300 iterations at learning rate 0.01 with a cosine schedule. In the spanning-tree initialization, the code calls PnP only for images that the tree propagation did not already pose. The supplement’s ablation (§F, Fig. 8) reports that fewer iterations change the result little and that an overall error of about 2 mm on a DTU scene of 49 images, with each image paired to its 10 nearest neighbours (490 pairs), is reached within 30 seconds on one H100.

Performance

All DUSt3R results below use one model, “DUSt3R 512” (ViT-L encoder, ViT-B decoder, DPT head), applied without fine-tuning to any benchmark; the paper calls this a zero-shot setting. Test images are resized to 512 pixels in the largest dimension.

Map-free relocalization. The task is to localize each frame of a video in metric space relative to a single reference image of the scene (130 test scenes, ground truth held privately). DUSt3R’s matches give the relative pose through an essential matrix with RANSAC in the benchmark’s code, and metric scale comes from a DPT model fine-tuned on KITTI, as for most other methods in the table. On the test set (Table 1), DUSt3R has the best VCRE (virtual correspondence reprojection error) values, a reprojection error of 115.8 px, precision of 50.3% and AUC of 0.697, and a median pose error of 0.98 m and 7.1°, against 1.23 m and 11.1° for RoMa. Its pose precision (21.4%) and AUC (0.393) are below RoMa’s (22.8%, 0.407), although the text says that DUSt3R outperforms all methods. The MASt3R paper’s Table 2 lists DUSt3R on the same test set with slightly different values (116.0 px, 0.97 m, 21.6%, 0.394).

Multi-view pose. On CO3Dv2 (41 categories) and RealEstate10K (1.8K test clips), 10 frames are drawn per sequence and all 45 pairs are evaluated with relative rotation and translation accuracy at 15° (RRA@15, RTA@15) and the mean average accuracy up to 30°, mAA(30) (Table 2, right). With global alignment DUSt3R reaches 96.2 RRA@15, 86.8 RTA@15 and 76.7 mAA(30) on CO3Dv2 and 67.7 mAA(30) on RealEstate10K; with PnP, 94.3, 88.4, 77.2 and 61.2; PoseDiffusion has 80.5, 79.8, 66.5 and 48.0, and COLMAP with SuperPoint and SuperGlue 36.1, 27.3, 25.3 and 45.2 (the supplement’s Table 1 gives its RRA@15 at 10 frames as 31.6). The text says that global alignment gives the best overall results; in the table, PnP has the higher RTA@15 and mAA(30) on CO3Dv2. DUSt3R’s training data include a small part of CO3Dv2 (50 sequences per category, under 7% of its training set) and no RealEstate10K (supplementary §C).

Monocular depth. Feeding $f(I, I)$ and reading $z$ (Table 2, left; AbsRel and $\delta_{1.25}$ accuracy), DUSt3R 512 scores 6.50 and 94.09 on NYU Depth v2 and 10.74 and 86.60 on KITTI, against 5.40 and 96.54 and 9.45 and 89.27 for the supervised DPT-BEiT. The paper describes it as on par with supervised baselines and better than the self-supervised ones. The same table ablates pretraining: at 224 pixels without CroCo initialization, the NYU AbsRel is 14.51, against 10.28 with it.

Multi-view depth. With no camera poses or depth ranges and predictions aligned to ground truth by the median depth (Table 3, left), DUSt3R 512 averages a relative error of 4.73 and an inlier ratio ( $\tau$ at 1.03) of 64.52 over KITTI, ScanNet, ETH3D, DTU and Tanks and Temples, with the best ETH3D values in the table (2.91 and 76.91). The Robust MVD baseline, given ground-truth poses, averages 6.3 and 56.0. ScanNet is not a zero-shot test, since its training split is part of DUSt3R’s Habitat data.

DTU reconstruction. After global alignment, with the predictions aligned to the ground-truth frame by fixing the camera parameters in Eq. 5, DUSt3R 512 reaches an accuracy of 2.677 mm, a completeness of 0.805 mm and an overall error of 1.741 mm on DTU (Table 3, right), against 0.295 mm overall for GeoMVSNet and 0.578 mm for Gipuma, both with ground-truth cameras. The authors note that these methods use known poses, train on DTU where applicable and triangulate with sub-pixel accuracy, while DUSt3R regresses.

Focal length. On 1,000 randomly sampled unseen test images from Habitat, BlendedMVS and CO3D, the average error of the field of view computed from the read-out focal is 4.13°, 3.40° and 1.88° with one image and 2.09°, 2.61° and 1.62° with two (supplementary Table 4); CO3D’s reference focals are themselves COLMAP estimates.

Visual localization. Used as a matcher against retrieved database images with known query intrinsics, DUSt3R 512 gives median errors on 7Scenes and Cambridge Landmarks comparable to feature-matching and scene-coordinate methods (supplementary Table 2), for example 3 cm and 0.97° on Chess. With unknown query focal, scaling by the database image’s ground-truth pointmap worked on 7Scenes but gave much larger errors on Cambridge, whose ground-truth pointmaps are sparse (supplementary Table 3).

Strengths

No calibration or poses. Reconstruction, focal lengths and poses come from images alone, including pairs with little or no overlap and nearly opposite viewpoints (supplementary Figs. 5 and 6).
One model, many outputs. Depth, intrinsics, relative and absolute pose, matches and dense multi-view reconstruction are all read from the same pointmaps, without task-specific training.
Robustness to wide baselines. On CO3Dv2, where the MASt3R paper notes that many pairs have wide baselines, both of DUSt3R’s read-outs beat the learned and SfM baselines by large margins (Table 2).
Simple multi-view fusion. Global alignment is a small 3D objective solved by gradient descent from a spanning-tree initialization, without reprojection errors.

Limitations

Regression accuracy. The paper attributes its DTU gap to regression, which it calls less accurate than sub-pixel triangulation with explicit cameras (§4.5). MASt3R’s authors call DUSt3R’s matches relatively imprecise for the same reason.
Scale and fixed resolution. DUSt3R’s pointmaps are up to scale, so metric results need another source of scale (a depth network on Map-free, ground truth for localization). The ViT works at 512 pixels in the largest dimension; MASt3R notes that transformers do not yet generalize to larger test-time resolutions and that downscaling loses accuracy (§3.4).
Pairwise cost. Every edge needs its own network pass, and the MASt3R supplement (§5) notes that both models scale poorly to large collections, with cost quadratic in the number of images unless pairs are pruned, for example by retrieval.
Camera assumptions. The focal read-out assumes square pixels and a centred principal point; training crops keep the principal point centred, and the supplement reports little effect when it is not exactly centred at test time. Each camera ray is assumed to hit one surface.
Learned confidence. The confidence is trained without supervision and only as well as the training distribution allows; it is also what global alignment weights by.
Static scenes and appearance change. The training data are mostly static and geometrically consistent. MASt3R’s supplement reports failures under large seasonal or scene changes (summer against winter, added scaffolding), and that its metric depth, being data-driven and reliant on priors, offers little guarantee.

Historical Importance

DUSt3R moved two-view reconstruction from estimating cameras first to regressing 3D points in a shared frame and reading the cameras off afterwards. Its paper presents it as the first pipeline to reconstruct end to end from uncalibrated, unposed images while unifying the monocular and binocular cases. The pointmap output, the confidence-aware loss and the CroCo-initialized two-branch transformer became the base of MASt3R, which starts from the DUSt3R checkpoint, and later feed-forward reconstruction models compare with it: VGGT, which predicts cameras and pointmaps for many views at once, is compared with DUSt3R on DTU by its authors (see 3D reconstruction). On Map-free relocalization, MASt3R’s paper describes DUSt3R as the method at the top of the leaderboard, although it was not trained for matching.

Variants

MASt3R

MASt3R keeps DUSt3R’s encoder, decoders and 3D head and is initialized from the public DUSt3R checkpoint.

Metric pointmaps. Scale invariance is unwanted where metric output is needed, as in map-free localization. When the ground truth is metric, MASt3R sets the predicted pointmap’s normalization to the ground truth’s, $z := \bar z$ , so that $\ell_{\text{regr}} = \lVert X_i^{v,1} - \bar X_i^{v,1} \rVert / \bar z$ ; otherwise it keeps DUSt3R’s loss. Ten of its 14 training datasets have metric ground truth (§4.1).

Matching head. A second head per branch, a two-layer MLP with a GELU, maps the concatenated encoder and decoder tokens to dense local features $D^1, D^2 \in \mathbb{R}^{H \times W \times d}$ with $d = 24$ , normalized to unit length. It is trained with an InfoNCE loss over the ground-truth correspondences $\hat{\mathcal{M}}$ , the reciprocal matches between the ground-truth pointmaps (Eq. 7):

\mathcal{L}_{\text{match}} = -\sum_{(i,j) \in \hat{\mathcal{M}}} \left[ \log \frac{s_\tau(i, j)}{\sum_{k \in \mathcal{P}^1} s_\tau(k, j)} + \log \frac{s_\tau(i, j)}{\sum_{k \in \mathcal{P}^2} s_\tau(i, k)} \right],

a cross-entropy over the candidate pixels of each image, so that only the exact pixel is rewarded, not a nearby one as in regression. The total loss is $\mathcal{L}_{\text{conf}} + \beta \mathcal{L}_{\text{match}}$ with $\beta = 1$ . The paper prints the similarity as $s_\tau(i, j) = \exp(-\tau D_i^{1\top} D_j^2)$ with temperature $\tau = 0.07$ (supplement §6); the official InfoNCE instead divides the dot product by the temperature without the minus sign, $\exp(D_i^{1\top} D_j^2 / \tau)$ , the usual form, with a default temperature of 0.07, and the README’s training command uses a temperature of 0.05 inside a confidence-weighted matching loss (ConfMatchingLoss) that is added with weight 0.075. Per pair, 4,096 ground-truth correspondences are sampled, padded with false ones when too few exist. Training samples 650,000 pairs per epoch for 35 epochs at learning rate $10^{-4}$ with cosine decay (supplementary Table 3); the README’s command runs 50 epochs.

Fast reciprocal matching. Mutual nearest neighbours over all pixels cost $O(W^2H^2)$ . MASt3R starts instead from $k$ pixels on a grid in the first image, maps them to their nearest neighbours in the second image and back, keeps those that return to their start (a cycle, hence a reciprocal match), and iterates on the rest. The supplement proves that every start converges to a reciprocal match, since each pixel’s single nearest-neighbour edge makes every connected subgraph an arborescence with one cycle at its root, and that at most $k$ matches result. The cost is $O(kWH)$ . With $k = 3000$ it was 64 times faster than full matching on Map-free and gave a higher VCRE AUC (Fig. 2); the supplement explains the gain by a sampling bias towards large convergence basins, which spreads matches more evenly over the image, and shows that random subsampling of the full set instead degrades results badly (supplementary Fig. 8). Nearest neighbours use k-d trees for 3D points and the FAISS library for 24-dimensional features.

Coarse-to-fine matching. For images larger than the network’s 512 pixels, MASt3R first matches downscaled images, then tiles each full-resolution image into 512-pixel windows overlapping by 50%, greedily selects window pairs until they cover 90% of the coarse matches, matches each window pair and maps the matches back.

Results. On the Map-free validation set (Table 1), matching MASt3R’s features instead of its 3D points raised the VCRE AUC from 0.732 to 0.752, training with the matching loss alone gave 0.744 but a median rotation error of 10.8° against 3.0°, and using MASt3R’s own metric depth instead of DPT raised the AUC to 0.934. On the test set (Table 2), MASt3R with its own depth reached a VCRE precision of 79.3%, an AUC of 0.933 and a median error of 0.36 m and 2.2°, and direct regression (PnP on $X^{2,1}$ , without matching) an AUC of 0.941. The text calls this a 30-point improvement over the second-best published method, LoFTR with KBR depth (0.634); the table also lists FAR (0.680), RoMa (0.669), Mickey (0.748) and DUSt3R (0.697) above it. On CO3Dv2 and RealEstate10K with pairwise predictions only (Table 3), MASt3R reaches 94.6 RRA@15, 91.9 RTA@15 and 81.8 mAA(30) on CO3Dv2 and 76.4 mAA(30) on RealEstate10K. On Aachen Day-Night and InLoc it is competitive with or better than keypoint matchers when 20 or 40 database images are retrieved, while direct regression localizes poorly in these larger scenes (Table 4). On DTU, triangulating its full-resolution matches with the ground-truth cameras, which here are used only for triangulation, gives an accuracy of 0.403 mm, a completeness of 0.344 mm and an overall error of 0.374 mm (Table 3, right), a different protocol from DUSt3R’s 1.741 mm without cameras; coarse matching alone gives 0.622 mm overall (supplementary Table 1).

Later Models

As of November 2026 the DUSt3R README points to MASt3R, to Pow3R (DUSt3R conditioned on known depth, focal length or poses) and to MUSt3R (multi-view prediction without global alignment), and the MASt3R repository includes MASt3R-SfM’s retrieval and sparse global alignment; VGGT and other feed-forward 3D models are compared in 3D reconstruction.

Implementations

As of November 2026, the official repositories are naver/dust3r and naver/mast3r, read at commits 4c24a6e and f5209af. Both READMEs state that the code “is distributed under the CC BY-NC-SA 4.0 License”, and the source headers say “non-commercial use only”. The DUSt3R README offers three checkpoints, DUSt3R_ViTLarge_BaseDecoder_224_linear, DUSt3R_ViTLarge_BaseDecoder_512_linear and DUSt3R_ViTLarge_BaseDecoder_512_dpt, through a Hugging Face integration or a NAVER LABS download server, and asks users to agree to the licences of all the public training datasets and base checkpoints in addition to CC BY-NC-SA 4.0. The MASt3R README offers MASt3R_ViTLarge_BaseDecoder_512_catmlpdpt_metric and a retrieval model, and says that the Map-free dataset’s licence “in particular is very restrictive”; its CHECKPOINTS_NOTICE reproduces the licence terms of the datasets behind the checkpoints. The MASt3R README says that not all its training datasets were released. No checkpoint or dataset was downloaded for this article.

The repositories provide an interactive demo, the pairwise inference API, the global_aligner with PointCloudOptimizer (dense global alignment, initialized with init="mst"), training scripts for the released datasets, and in MASt3R fast_reciprocal_NNs and the sparse global alignment.

Read-Out and Global Alignment Without a Network

The script below shows the mechanics of §3.3 and §3.4 on simulated network outputs; no network is trained or loaded. It ray-casts a synthetic room (6 m by 6 m by 3 m, with a cube and a sphere) into five $64 \times 48$ cameras on an arc 2.2 m from the centre, with focal lengths of 44 to 56 pixels and the principal point at $(W/2, H/2)$ on the integer pixel grid, the official code’s convention. Cameras use OpenCV axes ( $x$ right, $y$ down, $z$ forward); depth is the $z$ coordinate; the relative pose maps camera-0 coordinates to camera-1 coordinates; and global alignment estimates camera-to-world poses, as the official code stores them. The optical axes of the pairs are 28° to 107° apart.

For every ordered pair $(n, m)$ of the 20, the script builds what DUSt3R would output: the ground-truth $\bar X^{n,n}$ and $\bar X^{m,n}$ with Gaussian noise of standard deviation $\rho$ times the depth, where $\rho$ grows with the angle between the cameras (about 1.9% to 4.6% for the first view, 1.5 times that for the second), an outlier disc of radius 7 pixels (at most 5% of the pixels) displaced by 30% of the depth in a random direction, and a random scale $e^{\mathcal{N}(0, 0.3^2)}$ per pass. The confidence is the minimizer $C^\ast = \max(1, \alpha/\mathbb{E}[\ell])$ derived above, with $\mathbb{E}[\ell]$ the expected normalized error of the pixel: an idealized confidence that knows its own error. Real confidences are learned and imperfect, a choice the sources leave open, so the script also runs global alignment with the same confidence values shuffled across all pixels, which keeps their distribution and removes their link to error.

Part 1 compares four focal estimators on the 20 first-view pointmaps. Part 2 recovers the relative pose of cameras 0 and 1 from the passes $(I^0, I^1)$ and $(I^1, I^0)$ by the two Procrustes routes. Part 3 implements Eq. 5 with the camera parametrization in PyTorch 2.14.1 (NumPy 2.5.4, SciPy 1.18.1), with the official optimizer’s defaults described above (Adam, 300 steps, learning rate 0.01, cosine decay). Three weightings are compared: none, $C$ (the paper’s Eq. 5) and $\log C$ (the code’s default). The initialization follows the code’s spanning tree, except that each image’s focal comes from its highest-scoring edge and its pose from Procrustes of its own pointmap onto its propagated points, rather than by RANSAC-PnP. Errors are measured after a least-squares similarity alignment of all estimated points to the ground truth. The --clean option removes noise, outliers and scales (with $C = 2$ , so that $\log C$ is not zero), and was run before the main run to check that every estimator and every weighting recovers the ground truth; with it, every error is zero to the printed precision at initialization and at most 0.03 cm, 0.001° and 0.01% after Adam. A run takes about 13 seconds on a laptop CPU with one thread. After the first run, the seed was made an optional argument (default 0, which reproduces the first run exactly) so that seeds 1 to 4 could also be run. In fact-checking, the initial pair translations were corrected to the official code’s form (translation divided by the pair scale), since the earlier line applied the scale normalization twice, and all runs were repeated.

import sys
import numpy as np
import torch
from scipy.spatial.transform import Rotation

CLEAN = "--clean" in sys.argv          # pilot: no noise, no outliers, unit scales, C = 2
SEED = int(next((a for a in sys.argv[1:] if a.isdigit()), 0))     # seed as an optional argument
torch.set_num_threads(1)
torch.manual_seed(SEED)
rng = np.random.default_rng(SEED)
W, H, ALPHA = 64, 48, 0.2
cx, cy = W / 2, H / 2                   # principal point at (W/2, H/2), as in the official code
u, v = np.meshgrid(np.arange(W, dtype=float), np.arange(H, dtype=float))

# ---------- synthetic scene: a room (x, y in [-3, 3] m, z in [0, 3] m), a cube and a sphere ----------
def cast(C, d):
    """Distance along rays C + lam d to the first surface (d: N x 3, world frame)."""
    lo, hi = np.array([-3.0, -3.0, 0.0]), np.array([3.0, 3.0, 3.0])
    d = np.where(d == 0, 1e-12, d)                                               # no exactly axis-parallel rays
    with np.errstate(divide="ignore", invalid="ignore"):
        lam = np.min(np.where(d > 0, (hi - C) / d, (lo - C) / d), axis=1)        # room walls (inside)
        a, b = (np.array([-0.5, -0.5, 0.0]) - C) / d, (np.array([0.5, 0.5, 1.0]) - C) / d
        t0, t1 = np.max(np.minimum(a, b), axis=1), np.min(np.maximum(a, b), axis=1)
    lam = np.where((t0 < t1) & (t0 > 0) & (t0 < lam), t0, lam)                    # cube
    oc = C - np.array([0.9, -0.7, 0.5])
    bq, cq = np.sum(d * oc, 1), oc @ oc - 0.5 ** 2
    disc = bq ** 2 - np.sum(d * d, 1) * cq
    ts = (-bq - np.sqrt(np.maximum(disc, 0))) / np.sum(d * d, 1)
    return np.where((disc > 0) & (ts > 0) & (ts < lam), ts, lam)                  # sphere

def look_at(C, target):
    z = (target - C) / np.linalg.norm(target - C)
    x = np.cross(z, [0.0, 0.0, 1.0]); x /= np.linalg.norm(x)
    return np.stack([x, np.cross(z, x), z])          # world-to-camera rotation, x right, y down, z forward

focals = np.array([44.0, 50.0, 56.0, 48.0, 52.0])
cams = []                                             # (R world-to-camera, centre C)
for az in np.radians([-60, -30, 0, 30, 60]):
    C = np.array([2.2 * np.cos(az), 2.2 * np.sin(az), 1.5])
    cams.append((look_at(C, np.array([0.0, 0.0, 0.6])), C))
N = len(cams)

Xcam = []                                             # ground-truth pointmap of each image in its own frame
for n, (R, C) in enumerate(cams):
    dc = np.stack([(u - cx) / focals[n], (v - cy) / focals[n], np.ones_like(u)], -1).reshape(-1, 3)
    Xcam.append(dc * cast(C, dc @ R)[:, None])         # z of dc is 1, so lam is the depth
to_world = lambda n, X: X @ cams[n][0] + cams[n][1]
to_cam = lambda n, Xw: (Xw - cams[n][1]) @ cams[n][0].T
Xw_gt = [to_world(n, Xcam[n]) for n in range(N)]

# ---------- simulated DUSt3R outputs for every ordered pair e = (n, m), both in camera n's frame ----------
def disc_mask():
    c = rng.uniform([0, 0], [W, H])
    return ((u - c[0]) ** 2 + (v - c[1]) ** 2 <= 7 ** 2).ravel()

edges, pred = [], {}
for n in range(N):
    for m in range(N):
        if n == m:
            continue
        ang = np.degrees(np.arccos(np.clip(cams[n][0][2] @ cams[m][0][2], -1, 1)))
        rho = 0.01 * (1 + ang / 30)                                     # noise grows with the angle
        gt = [Xcam[n], to_cam(n, Xw_gt[m])]
        zbar = np.mean(np.linalg.norm(np.vstack(gt), axis=1))           # the paper's norm(.)
        s_e = 1.0 if CLEAN else np.exp(rng.normal(0, 0.3))
        out = []
        for k, (Xg, depth, r) in enumerate([(gt[0], Xcam[n][:, 2], rho), (gt[1], Xcam[m][:, 2], 1.5 * rho)]):
            sig = r * depth
            if CLEAN:
                out += [s_e * Xg, np.full(H * W, 2.0)]
                continue
            X = Xg + rng.normal(size=Xg.shape) * sig[:, None]
            El = np.sqrt(8 / np.pi) * sig / zbar                           # expected regression error
            o = disc_mask()                                                # outlier disc
            dirs = rng.normal(size=(o.sum(), 3)); dirs /= np.linalg.norm(dirs, axis=1, keepdims=True)
            X[o] = Xg[o] + 0.3 * depth[o, None] * dirs
            El[o] = 0.3 * depth[o] / zbar
            out += [s_e * X, np.maximum(1.0, ALPHA / El)]                  # loss-optimal confidence
        pred[(n, m)] = dict(Xn=out[0], Cn=out[1], Xm=out[2], Cm=out[3], s=s_e)
        edges.append((n, m))

# ---------- part 1: focal length from X^{n,n} ----------
pix = np.stack([u.ravel() - cx, v.ravel() - cy], 1)

def focal(X, mode, C=None):
    q = X[:, :2] / X[:, 2:3]
    pq, qq = np.sum(pix * q, 1), np.sum(q * q, 1)
    if mode == "median":
        with np.errstate(divide="ignore", invalid="ignore"):
            votes = np.concatenate([pix[:, 0] * X[:, 2] / X[:, 0], pix[:, 1] * X[:, 2] / X[:, 1]])
        return np.nanmedian(votes)
    c = np.ones(len(X)) if C is None else C
    f = np.mean(c * pq) / np.mean(c * qq)                                   # L2 closed form
    if mode == "l2":
        return f
    for _ in range(10):                                                     # Weiszfeld (IRLS on L1)
        w = c / np.maximum(np.linalg.norm(pix - f * q, axis=1), 1e-8)
        f = np.mean(w * pq) / np.mean(w * qq)
    return f

print("1. focal from X^{n,n}, 20 pointmaps: |error| median / max (%)")
for name, mode, use_c in [("L2 closed form", "l2", False), ("Weiszfeld", "weiszfeld", False),
                          ("Weiszfeld, weights C", "weiszfeld", True), ("median of votes", "median", False)]:
    err = [100 * abs(focal(pred[e]["Xn"], mode, pred[e]["Cn"] if use_c else None) / focals[e[0]] - 1)
           for e in edges]
    print(f"   {name:22s} {np.median(err):6.3f} / {np.max(err):6.3f}")
e0 = pred[(0, 1)]
print("   pair (0,1), weighted Weiszfeld: f = %.3f; pointmap x 3.7: f = %.3f; true 44"
      % (focal(e0["Xn"], "weiszfeld", e0["Cn"]), focal(3.7 * e0["Xn"], "weiszfeld", e0["Cn"])))

# ---------- part 2: relative pose of cameras 0 and 1 from the passes (I0, I1) and (I1, I0) ----------
def procrustes(P, Q, w):
    """Weighted Umeyama: s, R, t minimizing sum w ||s R p + t - q||^2."""
    w = w / w.sum()
    p0, q0 = w @ P, w @ Q
    Hm = (P - p0).T @ ((Q - q0) * w[:, None])
    U, S, Vt = np.linalg.svd(Hm)
    d = np.sign(np.linalg.det(Vt.T @ U.T))
    R = Vt.T @ np.diag([1, 1, d]) @ U.T
    s = (S[0] + S[1] + d * S[2]) / (w @ np.sum((P - p0) ** 2, 1))
    return s, R, q0 - s * R @ p0

ang_deg = lambda a, b: np.degrees(np.arccos(np.clip(a @ b / np.linalg.norm(a) / np.linalg.norm(b), -1, 1)))
R10 = cams[1][0] @ cams[0][0].T                                             # camera-0 to camera-1 frame
t10 = -cams[1][0] @ cams[1][1] - R10 @ (-cams[0][0] @ cams[0][1])
a, b = pred[(0, 1)], pred[(1, 0)]
print("2. relative pose 0 -> 1 (true rotation %.1f deg); scale ratio s(1,0)/s(0,1) = %.4f" % (
    np.degrees(Rotation.from_matrix(R10).magnitude()), b["s"] / a["s"]))
for name, P, Q, w in [("X^{0,0} <-> X^{0,1}", a["Xn"], b["Xm"], a["Cn"] * b["Cm"]),
                      ("X^{1,0} <-> X^{1,1}", a["Xm"], b["Xn"], a["Cm"] * b["Cn"])]:
    s, R, t = procrustes(P, Q, w)
    print(f"   {name}: rotation error {np.degrees(Rotation.from_matrix(R @ R10.T).magnitude()):.3f} deg, "
          f"translation direction error {ang_deg(t, t10):.3f} deg, sigma {s:.4f}")

# ---------- part 3: global alignment (Eq. 5) with the camera parametrization ----------
def init_mst(C_of):
    """Spanning-tree initialization as in the official code: chi^n (world points) and focals."""
    score = {e: C_of[e][0].mean() + C_of[e][1].mean() for e in edges}
    best = {}
    for (n, m), sc in score.items():                                        # one ordered edge per pair
        k = (min(n, m), max(n, m))
        if k not in best or sc > score[best[k]]:
            best[k] = (n, m)
    comp, tree = list(range(N)), []                                          # Kruskal, highest score first
    for e in sorted(best.values(), key=lambda e: -score[e]):
        if comp[e[0]] != comp[e[1]]:
            old = comp[e[1]]; comp = [comp[e[0]] if c == old else c for c in comp]; tree.append(e)
    chi = [None] * N
    n, m = tree[0]
    chi[n], chi[m] = pred[tree[0]]["Xn"].copy(), pred[tree[0]]["Xm"].copy()
    todo = tree[1:]
    while todo:
        e = todo.pop(0); n, m = e
        if chi[n] is not None and chi[m] is None:
            s, R, t = procrustes(pred[e]["Xn"], chi[n], C_of[e][0]); chi[m] = s * pred[e]["Xm"] @ R.T + t
        elif chi[m] is not None and chi[n] is None:
            s, R, t = procrustes(pred[e]["Xm"], chi[m], C_of[e][1]); chi[n] = s * pred[e]["Xn"] @ R.T + t
        else:
            todo.append(e)
    f0 = []
    for n in range(N):                                                      # focal from the best edge with n first
        e = max([e for e in edges if e[0] == n], key=lambda e: score[e])
        f0.append(focal(pred[e]["Xn"], "weiszfeld"))
    return chi, np.array(f0)

def quat_to_R(q):
    q = q / q.norm(dim=-1, keepdim=True)
    x, y, z, w = q.unbind(-1)
    return torch.stack([1 - 2 * (y * y + z * z), 2 * (x * y - z * w), 2 * (x * z + y * w),
                        2 * (x * y + z * w), 1 - 2 * (x * x + z * z), 2 * (y * z - x * w),
                        2 * (x * z - y * w), 2 * (y * z + x * w), 1 - 2 * (x * x + y * y)], -1).view(*q.shape[:-1], 3, 3)

slog1p = lambda x: np.sign(x) * np.log1p(np.abs(x))
sexpm1 = lambda x: torch.sign(x) * torch.expm1(x.abs())

def global_alignment(C_of, weighting, niter=300, lr=0.01):
    chi, f0 = init_mst(C_of)
    dt = torch.float64
    tw = lambda x: torch.tensor(x, dtype=dt)
    # pair poses (cam-to-world of camera n's frame) and log-scales, by Procrustes of X^{n,e} onto chi^n
    pw = []
    for e in edges:
        s, R, t = procrustes(pred[e]["Xn"], chi[e[0]], C_of[e][0])
        pw.append((s, R, t))
    base = np.log(0.5) - np.mean([np.log(p[0]) for p in pw])                 # code: mean log-scale -> log 0.5
    pw_q = torch.nn.Parameter(tw(np.array([Rotation.from_matrix(p[1]).as_quat() for p in pw])))
    pw_t = torch.nn.Parameter(tw(np.array([slog1p(p[2] / p[0]) for p in pw])))  # code: T / s, scaled in loss()
    pw_ls = torch.nn.Parameter(tw(np.array([np.log(p[0]) for p in pw])))
    chi = [c * np.exp(base) for c in chi]
    # camera poses by Procrustes of X^{n,e} onto chi^n (rigid part), depth = z of chi^n in that camera
    im_q, im_t, logd = [], [], []
    for n in range(N):
        e = max([e for e in edges if e[0] == n], key=lambda e: C_of[e][0].mean() + C_of[e][1].mean())
        s, R, t = procrustes(pred[e]["Xn"], chi[n], C_of[e][0])
        im_q.append(Rotation.from_matrix(R).as_quat()); im_t.append(slog1p(t))
        logd.append(np.log(np.maximum(((chi[n] - t) @ R)[:, 2], 1e-6)))
    im_q, im_t = torch.nn.Parameter(tw(np.array(im_q))), torch.nn.Parameter(tw(np.array(im_t)))
    logd = torch.nn.Parameter(tw(np.array(logd)))
    logf = torch.nn.Parameter(tw(20 * np.log(f0)))
    grid = tw(pix)
    Xn = torch.stack([tw(pred[e]["Xn"]) for e in edges]); Xm = torch.stack([tw(pred[e]["Xm"]) for e in edges])
    trf = {"none": lambda c: np.ones_like(c), "C": lambda c: c, "log C": np.log}[weighting]
    wn = torch.stack([tw(trf(C_of[e][0])) for e in edges]); wm = torch.stack([tw(trf(C_of[e][1])) for e in edges])
    ei, ej = torch.tensor([e[0] for e in edges]), torch.tensor([e[1] for e in edges])

    def world_points():
        d, f = logd.exp(), (logf / 20).exp()
        Xc = torch.cat([d[..., None] * grid[None] / f[:, None, None], d[..., None]], -1)
        return Xc @ quat_to_R(im_q).transpose(1, 2) + sexpm1(im_t)[:, None]

    def loss():
        chi_t = world_points()
        sc = (pw_ls - pw_ls.mean() + np.log(0.5)).exp()[:, None, None]
        R, t = quat_to_R(pw_q), sexpm1(pw_t)[:, None]
        an = sc * (Xn @ R.transpose(1, 2)) + sc * t; am = sc * (Xm @ R.transpose(1, 2)) + sc * t
        li = ((chi_t[ei] - an).norm(dim=-1) * wn).mean(1)
        lj = ((chi_t[ej] - am).norm(dim=-1) * wm).mean(1)
        return (li + lj).mean()

    def evaluate():
        with torch.no_grad():
            est = world_points().numpy().reshape(-1, 3)
            Rc = quat_to_R(im_q).numpy(); Cc = sexpm1(im_t).numpy(); f = (logf / 20).exp().numpy()
        s, R, t = procrustes(est, np.vstack(Xw_gt), np.ones(len(est)))
        perr = np.median(np.linalg.norm(s * est @ R.T + t - np.vstack(Xw_gt), axis=1))
        rerr = np.mean([np.degrees(Rotation.from_matrix((R @ Rc[n]) @ cams[n][0]).magnitude()) for n in range(N)])
        cerr = np.mean([np.linalg.norm(s * R @ Cc[n] + t - cams[n][1]) for n in range(N)])
        return 100 * perr, rerr, 100 * cerr, 100 * np.mean(np.abs(f / focals - 1))

    res = {"init": evaluate()}
    params = [pw_q, pw_t, pw_ls, im_q, im_t, logd, logf]
    opt = torch.optim.Adam(params, lr=lr, betas=(0.9, 0.9))
    for it in range(niter):
        for g in opt.param_groups:                                           # cosine schedule to 1e-6
            g["lr"] = 1e-6 + (lr - 1e-6) * (1 + np.cos(np.pi * it / niter)) / 2
        opt.zero_grad(); L = loss(); L.backward(); opt.step()
    res["final"] = evaluate()
    return res

C_true = {e: (pred[e]["Cn"], pred[e]["Cm"]) for e in edges}
allC = np.concatenate([np.concatenate(c) for c in C_true.values()])
perm = rng.permutation(allC)
C_shuf, k = {}, 0
for e in edges:
    C_shuf[e] = (perm[k:k + H * W], perm[k + H * W:k + 2 * H * W]); k += 2 * H * W

print("3. global alignment, 5 cameras, 20 ordered pairs, 300 Adam steps")
print("   confidence    weights   point err (cm)  rot err (deg)  centre err (cm)  focal err (%)")
for cname, C_of in [("informative", C_true), ("shuffled", C_shuf)]:
    for wname in ["none", "C", "log C"]:
        r = global_alignment(C_of, wname)
        if wname == "none":
            print("   %-12s  %-8s %10.2f %14.3f %15.2f %14.2f" % (cname, "(init)", *r["init"]))
        print("   %-12s  %-8s %10.2f %14.3f %15.2f %14.2f" % (cname, wname, *r["final"]))

Output (seed 0):

1. focal from X^{n,n}, 20 pointmaps: |error| median / max (%)
   L2 closed form          2.212 /  3.892
   Weiszfeld               0.886 /  2.644
   Weiszfeld, weights C    0.836 /  2.404
   median of votes         0.628 /  1.484
   pair (0,1), weighted Weiszfeld: f = 43.778; pointmap x 3.7: f = 43.778; true 44
2. relative pose 0 -> 1 (true rotation 30.0 deg); scale ratio s(1,0)/s(0,1) = 0.7302
   X^{0,0} <-> X^{0,1}: rotation error 0.114 deg, translation direction error 0.310 deg, sigma 0.7277
   X^{1,0} <-> X^{1,1}: rotation error 0.048 deg, translation direction error 0.365 deg, sigma 0.7261
3. global alignment, 5 cameras, 20 ordered pairs, 300 Adam steps
   confidence    weights   point err (cm)  rot err (deg)  centre err (cm)  focal err (%)
   informative   (init)         4.92          0.127            3.34           0.63
   informative   none           2.16          0.041            0.64           0.18
   informative   C              1.99          0.039            0.44           0.13
   informative   log C          2.01          0.045            0.49           0.15
   shuffled      (init)         6.74          0.240            6.65           0.98
   shuffled      none           2.16          0.041            0.66           0.18
   shuffled      C              2.61          0.085            0.81           0.24
   shuffled      log C          2.39          0.064            0.68           0.21

Focal length. The closed-form least-squares focal is pulled by the outlier discs, whose points project far from their pixels, and is off by about 2% in the median; the Weiszfeld iterations, which minimize unsquared distances, cut that by more than half, and the paper’s confidence weights reduce it a little further. The median of per-pixel votes, the code’s other mode, is the most accurate of the four here, against our expectation before the run that the weighted Weiszfeld estimate would be. Over seeds 0 to 4 the order is the same in every run: L2 from 2.21% to 2.51%, Weiszfeld from 0.89% to 1.19%, weighted Weiszfeld from 0.84% to 0.99% and the median from 0.63% to 0.76%. Multiplying the pointmap by 3.7 leaves the estimate unchanged, because the read-out depends only on $x/z$ and $y/z$ , which is why DUSt3R’s unknown scale does not affect it.

Relative pose. Both routes give the 30° relative rotation to about a tenth of a degree and the direction of the baseline to about a third of a degree. They also return $\sigma$ , the ratio of the scales of the two passes, to within 0.6% of the true ratio, which is what global alignment’s pair scales must undo. The two routes use different pixels (image 0’s in one, image 1’s in the other) and so give slightly different answers from the same two passes; the Kabsch Algorithm article’s example covers how weighting and noise affect such closed-form fits.

Global alignment. The spanning-tree initialization propagates poses along only four of the ten pairs, and the optimization over all 20 ordered pairs halves the median point error and cuts the camera errors by a factor of about three or more. With informative confidences, weighting by $C$ gives the lowest point and centre errors in seed 0 and $\log C$ comes close; the paper’s and the code’s weightings differ little here. Over seeds 0 to 4, the median point error ranges from 1.98 to 1.99 cm with $C$ , 1.99 to 2.01 cm with $\log C$ and 2.14 to 2.18 cm without weights, and the centre error is lowest with $C$ , then $\log C$ , then no weights in every seed. The order of all three on the rotation and focal errors, which are fractions of a degree and of a percent, changes from seed to seed; in seed 0, against our expectation before the run that both weightings would beat no weights on every measure, $\log C$ has a slightly larger rotation error than no weights. In our reading, weighting by $C^\ast \propto 1/\mathbb{E}[\ell]$ down-weights both the outlier discs and the noisier wide-baseline pairs, while $\log C$ compresses the same ordering and gives zero weight to every pixel whose $C$ is 1, which by construction includes the outlier pixels deeper than two thirds of $\bar z$ . With shuffled confidences the weights carry no information. In seed 0 they make every measure worse than no weighting, and over the five seeds the point error is always worse (2.58 to 2.64 cm with $C$ , 2.37 to 2.41 cm with $\log C$ ). They also degrade the initialization, which chooses its tree and its Procrustes weights by confidence, yet unweighted alignment reaches nearly the same result from either initialization. The confidence is useful to the extent that it tracks the error, which a real network learns only approximately.