Learned Local Features

Learned local features are keypoint detectors and local descriptors whose functions are trained on image data instead of designed by hand. They do the job of SIFT or ORB: find points that can be found again in another image of the same scene, and summarize each one in a vector that can be compared with a distance. What changes is where the invariances come from. A hand-designed descriptor is robust to rotation because it is computed in a rotated frame; a learned one is robust to whatever changes its training pairs contained, such as day against night or one season against another. The family runs from networks that only describe patches cut out by a classical detector, such as L2-Net, HardNet and SOSNet, to networks that both detect and describe, such as SuperPoint and D2-Net, and later learned features such as R2D2, DISK and ALIKED. Learned matchers such as SuperGlue and LoFTR, which feature matching compares with classical matching, consume such features.

Definition

A learned local feature extractor is a parameterized function, usually a convolutional network, that maps an image, or a patch of it, to keypoint locations with scores, to descriptors, or to both, and whose parameters are fitted so that points in two images that show the same scene point are detected in both and receive nearby descriptors, while different points receive distant ones. The two halves can be learned separately (a learned descriptor on hand-detected patches, or a learned detector feeding a classical descriptor) or jointly in one network.

Intuition

A local feature has two jobs, and they pull in different directions. The detector must be repeatable: under a change of viewpoint or lighting, it must fire at the same scene points. The descriptor must be distinctive: among all the points detected in the other image, the right partner must be the nearest. A detector built from low-level image structure, a corner or a blob, looks at a small neighbourhood. D2-Net’s authors argue that this is why such detectors become unstable under strong appearance changes such as day against night, while descriptors, which look at larger patches, can still be matched there.

Learning lets both functions be fitted to the changes that matter in practice, if training pairs exhibit them. It also lets the two jobs share computation. In D2-Net the same feature maps serve as descriptors and as detector responses, so a point is detected where its own descriptor channels respond most strongly.

Repeatability and distinctiveness are still different properties. A row of identical windows on a facade gives a detector many well-defined corners that reappear in every photograph. Each corner is repeatable, but its descriptor looks like every other window’s, so the nearest descriptor is often the wrong window. This framing, which the example below measures, is our own reasoning; feature detection lists the same tension among its open problems.

Formal Definition

Let $I$ be an image. A dense extractor computes a tensor $F = \mathcal{F}(I) \in \mathbb{R}^{h \times w \times n}$. D2-Net reads it in two ways (Dusmanu et al., 2019, §3). As descriptors, the vector at each position is

$$\mathbf{d}_{ij} = F_{ij:} \in \mathbb{R}^n , \qquad \hat{\mathbf{d}}_{ij} = \mathbf{d}_{ij} / \lVert \mathbf{d}_{ij} \rVert_2 ,$$

compared with the Euclidean distance. As detectors, each channel is a response map $D^k = F_{::k}$, analogous, the authors note, to a difference-of-Gaussians map in SIFT or the corner score in Harris. The hard detection rule (their Eq. 3) keeps a position when its dominant channel has a local maximum there:

$$(i, j) \text{ is a detection} \iff D^k_{ij} \text{ is a local maximum of } D^k, \quad k = \arg\max_t D^t_{ij} .$$

For training, the rule is softened so that it can be differentiated: a soft local-maximum score $\alpha^k_{ij} = \exp(D^k_{ij}) / \sum_{(i', j') \in \mathcal{N}(i, j)} \exp(D^k_{i'j'})$ over the $3 \times 3$ neighbourhood, a ratio to the channel maximum $\beta^k_{ij} = D^k_{ij} / \max_t D^t_{ij}$, and a score $s_{ij} = \gamma_{ij} / \sum_{i'j'} \gamma_{i'j'}$ with $\gamma_{ij} = \max_k \alpha^k_{ij} \beta^k_{ij}$ (Eqs. 4–7).

The two halves are judged by different measures. With a ground-truth map $H$ between two images, such as a homography, the repeatability of a detector is the number of image-1 points with a detection in image 2 within a distance $\varepsilon$ of their mapped position, divided by the smaller of the two point counts in the shared region (feature detection gives the details). The descriptor is judged at the correspondences: whether the true partner is the nearest descriptor, or how it ranks among all candidates. Training losses for descriptors, such as the triplet margin ranking loss, make these ranks small; triplet loss and metric learning cover the losses and the mining of negatives.

Properties

Detect-then-describe against describe-and-detect. D2-Net’s paper sorts methods by where detection happens. In the classical two-stage pipeline, a detector picks points from low-level structure and a descriptor is computed on a patch around each; either stage, or both, can be learned. In describe-and-detect, a single network computes dense features first and detection is postponed to them. The authors report that this made their keypoints more stable under strong appearance changes but less accurately localized, since detection works on higher-level features at a quarter of the input resolution at test time (§1, §4.3, §5.1).

Localization against robustness. On HPatches image pairs, D2-Net’s mean matching accuracy (the share of mutual-nearest-neighbour matches whose reprojection error is below a threshold) was the best of the methods compared at thresholds of 6.5 pixels or more, and worse than detect-then-describe methods at stricter thresholds, which the authors attribute to those methods’ better-localized low-level detections (§5.1, Fig. 4). In the authors’ 3D reconstruction experiment, Lowe’s ratio test removed 20%–25% of D2-Net’s correct matches, because the training loss did not take that filter into account (§5.2).

Where the invariance comes from. A learned feature is invariant to what its training pairs varied. D2-Net was trained on pixel correspondences from MegaDepth reconstructions; its authors suspect that its weaker results on HPatches viewpoint sequences come from a bias in that data, where about 90% of training pairs differ in viewpoint by less than 20° (§5.1). Invariance and equivariance discusses the general point.

Sources of supervision. Patch descriptors learn from datasets of matching patches: L2-Net trained on the Brown dataset and on HPatches, where patches sharing a 3D point index match. D2-Net extracted 327,036 image pairs with dense correspondences from multi-view-stereo depth maps of MegaDepth, removing occluded pixels with a depth check (§4.2). SuperPoint, as feature detection describes, avoids manual labels by learning corners on synthetic shapes and aggregating detections over random homographic warps.

Cost. Dense extraction runs a network over the whole image. D2-Net’s authors list the need to extract descriptors densely as a drawback compared with classical sparse features, while noting that it takes a single forward pass; in their InLoc experiment, D2-Net features took about 120 MB per image against 5 MB for affine RootSIFT (Table 2).

The Family

Learned patch descriptors

L2-Net (Tian, Fan and Wu, 2017) maps a $32 \times 32$ patch to a 128-dimensional unit vector with a fully convolutional network, downsampling with stride-2 convolutions and ending in a local response normalization layer, so the output can replace SIFT in any pipeline that matches with the L2 distance. Its training goal is relative: in a batch of $p$ matching pairs, every descriptor’s nearest neighbour should be its partner. The $p \times p$ distance matrix between the two halves of the batch is computed with one matrix product, and a softmax over each row and each column turns its diagonal into match probabilities whose log-likelihood is maximized. Two further terms penalize correlation between descriptor dimensions and supervise intermediate feature maps the same way. On the Brown dataset, the authors reported a mean false positive rate at 95% recall of 3.23% for L2-Net without augmentation, against 26.55% for SIFT (Table 1).

HardNet (Mishchuk et al., 2017), as SOSNet’s authors describe it, surpassed L2-Net by using a simple hinge triplet loss with “hardest-within-batch” mining. Feature descriptors summarizes it.

SOSNet (Tian et al., 2019) keeps the L2-Net architecture and its 128-dimensional unit descriptors, and adds a regularizer on second-order similarity: two matching descriptors should also have similar distances to the other descriptors in the batch. For each pair $i$, the regularizer takes the square root of the sum of the squared differences $\big(d(\mathbf{x}_i, \mathbf{x}_j) - d(\mathbf{x}^+_i, \mathbf{x}^+_j)\big)^2$ over the neighbouring pairs $j$ selected by a $K$-nearest-neighbour search with $K = 8$, averages it over the batch (Eqs. 2, 3 and 6), and is added with equal weight to a quadratic hinge triplet loss with hardest-in-batch negatives. On its own it cannot separate matches from non-matches, so the authors use it only as a regularizer (§3.3).

Learned detectors and joint networks

LIFT (Yi et al., 2016) is cited by D2-Net’s and LoFTR’s authors as an early method that learns both the detector and the descriptor within the detect-then-describe design. SuperPoint (DeTone et al., 2018) computes keypoints and descriptors for a whole image in one forward pass, with separate decoder heads; D2-Net’s authors contrast this with their own network, which shares all parameters between detection and description.

D2-Net (Dusmanu et al., 2019) takes VGG16 pretrained on ImageNet, truncated after conv4_3, and fine-tunes only that last layer. The loss extends the triplet margin ranking loss: each correspondence’s margin term, built from its descriptor distance and its hardest negative outside a small neighbourhood, is weighted by the product of the soft detection scores of the two points, so that the most distinctive correspondences receive the highest scores (Eq. 13). At test time, the last pooling layer is replaced by stride-1 average pooling and the following convolutions by dilated convolutions, keypoint positions are refined locally following SIFT’s approach, and an optional image pyramid at scales 0.5, 1 and 2 sums coarser feature maps into finer ones. The authors note that their approach already achieved state-of-the-art results without any training, using the ImageNet-pretrained features as they were (§1).

The paper and the official implementation differ on one point. Eq. (3) is the whole hard rule in the paper, but the official HardDetectionModule (lib/model_test.py at commit 0a3356c) also rejects edge-like maxima, with a test on a $3 \times 3$ finite-difference Hessian of each channel, of the same form as SIFT’s elimination of edge responses (trace squared over determinant at most $(r + 1)^2 / r$ with $r = 5$, and positive determinant). The detection score kept by lib/pyramid.py is the value of the detecting channel (divided by the level’s index plus one at later pyramid levels). The example below runs both variants.

R2D2 (Revaud et al., 2019), DISK (Tyszkiewicz, Fua and Trulls, 2020) and ALIKED (Zhao et al., 2023) are later learned local features. LoFTR’s authors count R2D2, D2-Net and DISK among the learned features that adopt the detector-based design. As of November 2026, ALIKED can be extracted in COLMAP since version 4.0, and ALIKED and DISK are part of the features module of OpenCV 5.0, as those articles describe.

Detector-free matching

LoFTR’s authors argue that detector-based methods, learned or not, struggle to find repeatable interest points in indistinctive regions, such as low-texture areas, and that their detector-free matcher finds good matches there and on repetitive patterns. Feature matching compares detector-free matchers with learned matchers such as SuperGlue.

Evaluation

Learned features are compared at three levels. Patch-level benchmarks, such as the Brown dataset and HPatches, score descriptors on patches extracted with known correspondences; binary descriptors explains their measures. Image-matching protocols detect and match full images related by known homographies: D2-Net’s HPatches protocol matches by mutual nearest neighbours and reports mean matching accuracy against a pixel threshold. Downstream tasks measure what the matches are for, such as structure from motion and visual localization. On the Aachen Day-Night local reconstruction task, where night-time queries are localized against models built from day-time images, D2-Net’s authors reported 44.9%, 66.3%, 88.8% and 100% of the 98 night queries within (0.5 m, 2°), (1 m, 5°), (5 m, 10°) and (10 m, 25°) for the fine-tuned single-scale model, against 36.7%, 54.1%, 72.5% and 81.6% for upright RootSIFT and 42.8%, 57.1%, 75.5% and 86.7% for SuperPoint (Fig. 5). The authors tuned the detection thresholds of HesAffNet + HardNet++, DELF and SuperPoint on the same task.

Results at one level do not carry to another. Feature matching describes pose-based benchmarks such as the Image Matching Challenge, and the SIFT paper article reports a comparison in which advanced hand-crafted descriptors still matched the learned ones of 2017 for reconstruction.

Examples

The example separates the two halves of a local feature on images with known ground truth. Nothing is trained: a fixed bank of filters stands in for a network’s feature maps, so that D2-Net’s detection rule can be applied to it exactly.

• Images, $512 \times 512$ with intensities in $[0, 1]$: a periodic texture, a $16 \times 16$ cell holding an L-shaped stroke and a small square, tiled $32 \times 32$ times, blurred and rescaled, and the bundled camera photograph.
• Feature map $F$: scale-normalized Gaussian derivatives $\sigma^{a+b}\, \partial^{a+b} (G_\sigma * I) / \partial y^a \partial x^b$ of first and second order at $\sigma = 2, 4, 8$, each split into its positive and negative parts, as a ReLU would: 30 nonnegative channels.
• Descriptor, the same for every detector: the channel vector $F_{ij:}$ at the keypoint pixel, unit-normalized as in D2-Net’s §3.1. Holding the descriptor fixed means that differences between rows come from where each detector puts its points.
• Detectors, each keeping its 300 highest-scoring points at least 24 pixels from the border: Harris (scikit-image, $k = 0.05$, $\sigma = 2$, $3 \times 3$ local maxima); D2-Net’s Eq. (3) on $F$ with $3 \times 3$ local maxima and a positive value, scored by the detecting channel’s value; and Eq. (3) plus the official code’s edge test.
• Pairs: five random homographies (seeds 0–4), each moving the four image corners by up to 25.6 pixels in $x$ and $y$; the second image is the first warped bilinearly. Positions are $x$ = column and $y$ = row, and a point at $\mathbf{x}$ in image 1 appears at $H(\mathbf{x})$ in image 2. Independent Gaussian noise of standard deviation 0.01 is added to both images.
• Measures: repeatability at $\varepsilon = 1.5$ pixels (the distance feature detection quotes) and 3 pixels; for each image-1 point that is repeatable, its partner is the nearest image-2 detection to $H(\mathbf{x})$, and nn@$\varepsilon$ is the share of these points whose partner is strictly nearer in descriptor distance than every other of image 2’s 300 points (ties count as failures); ratio@3 is the median of the partner’s descriptor distance over the nearest other point’s.

Before the experiment, the script checks its parts: that the texture repeats exactly with period 16, that the vectorized D2 rule agrees with a brute-force loop, that a warped Gaussian dot lands at $H(\mathbf{x})$ (which confirms the coordinate convention), and that with the identity map and no noise every detector is perfectly repeatable.

The planned hypotheses were: H1, on the texture, nn@3 stays below 0.5 for every detector, in every pair; H2, for every detector, nn@3 on camera exceeds nn@3 on the texture by at least 0.3; H3, for every detector, the texture’s repeatability at 3 pixels is no more than 0.1 below camera’s; H4, on camera, D2’s Eq. (3) gives points at least as distinctive (nn@3) as Harris, as D2-Net’s argument for describe-and-detect would suggest when transplanted to a fixed bank (our reasoning, not a claim the paper makes about this set-up).

# Learned local features: detection and description scored separately (pilot = planned run).
import os
for v in ("OMP_NUM_THREADS", "OPENBLAS_NUM_THREADS", "MKL_NUM_THREADS"):
    os.environ[v] = "1"
import numpy as np
from scipy import ndimage
from skimage import data, feature, transform

N, P, MARGIN, K = 512, 16, 24, 300
SIGMAS = (2.0, 4.0, 8.0)
ORDERS = ((0, 1), (1, 0), (0, 2), (1, 1), (2, 0))  # (row order, column order)
EPS = (1.5, 3.0)
JITTER = 0.05
NOISE = 0.01
SEEDS = range(5)


def texture():
    cell = np.zeros((P, P))
    cell[3:13, 4:6] = 1.0     # vertical stroke
    cell[11:13, 4:12] = 1.0   # foot of the L
    cell[4:7, 10:13] = 0.6    # small square
    img = ndimage.gaussian_filter(np.tile(cell, (N // P, N // P)), 1.0, mode="wrap")
    return 0.2 + 0.6 * img / img.max()


def camera():
    return data.camera() / 255.0


def bank(img):
    """Fixed bank: scale-normalized Gaussian derivatives, split into ReLU(+r), ReLU(-r)."""
    chans = []
    for s in SIGMAS:
        for oy, ox in ORDERS:
            r = s ** (oy + ox) * ndimage.gaussian_filter(img, s, order=(oy, ox), mode="reflect")
            chans += [np.maximum(r, 0.0), np.maximum(-r, 0.0)]
    return np.stack(chans, axis=-1)


def d2_detect(F, edge=None):
    """D2-Net hard detection, Eq. (3); edge=5 adds the official code's edge test."""
    is_dom = F == F.max(axis=-1, keepdims=True)
    is_loc = F == ndimage.maximum_filter(F, size=(3, 3, 1), mode="constant", cval=-np.inf)
    det = is_dom & is_loc & (F > 0)
    if edge is not None:
        kii = np.array([[0, 1.0, 0], [0, -2.0, 0], [0, 1.0, 0]])
        kij = 0.25 * np.array([[1.0, 0, -1.0], [0, 0, 0], [-1.0, 0, 1.0]])
        kjj = kii.T
        corr = lambda k: ndimage.correlate(F, k[:, :, None], mode="constant", cval=0.0)
        dii, dij, djj = corr(kii), corr(kij), corr(kjj)
        hdet, tr = dii * djj - dij * dij, dii + djj
        with np.errstate(divide="ignore", invalid="ignore"):
            not_edge = (tr * tr / hdet <= (edge + 1) ** 2 / edge) & (hdet > 0)
        det &= not_edge
    rows, cols, ch = np.nonzero(det)
    score = np.zeros(F.shape[:2])
    np.maximum.at(score, (rows, cols), F[rows, cols, ch])
    keep = score > 0
    return keep, score


def d2_bruteforce(F):
    h, w, c = F.shape
    out = np.zeros((h, w), bool)
    for i in range(h):
        for j in range(w):
            k = int(np.argmax(F[i, j]))
            if F[i, j, k] <= 0 or F[i, j, k] < F[i, j].max():
                continue
            nb = F[max(i - 1, 0):i + 2, max(j - 1, 0):j + 2, k]
            out[i, j] = F[i, j, k] >= nb.max()
    return out


def harris_detect(img):
    R = feature.corner_harris(img, method="k", k=0.05, eps=1e-6, sigma=2.0)
    keep = (R == ndimage.maximum_filter(R, size=3, mode="constant", cval=-np.inf)) & (R > 0)
    return keep, R


def top_k(keep, score):
    m = np.zeros_like(keep)
    m[MARGIN:N - MARGIN, MARGIN:N - MARGIN] = True
    r, c = np.nonzero(keep & m)
    order = np.argsort(-score[r, c], kind="stable")[:K]
    return np.stack([r[order], c[order]], axis=1)


def describe(F, kps):
    d = F[kps[:, 0], kps[:, 1]]
    return d / (np.linalg.norm(d, axis=1, keepdims=True) + 1e-12)


def detectors(img):
    F = bank(img)
    out = {}
    for name, (keep, score) in (("Harris", harris_detect(img)),
                                ("D2 Eq. (3)", d2_detect(F)),
                                ("D2 + edge test", d2_detect(F, edge=5))):
        kps = top_k(keep, score)
        out[name] = (kps, describe(F, kps))
    return out


def inside(xy):
    return np.all((xy >= MARGIN) & (xy <= N - 1 - MARGIN), axis=1)


def evaluate(k1, d1, k2, d2, H):
    """Repeatability at each eps, and rank/ratio of the true partner for repeatable points."""
    xy1 = k1[:, ::-1].astype(float)                  # (x = column, y = row)
    xy2 = k2[:, ::-1].astype(float)
    m1, m2 = H(xy1), H.inverse(xy2)
    s1, s2 = inside(m1), inside(m2)
    n = min(s1.sum(), s2.sum())
    gd = np.linalg.norm(m1[s1][:, None] - xy2[s2][None], axis=2)
    near = np.argmin(gd, axis=1)
    near_d = gd[np.arange(len(near)), near]
    idx2 = np.flatnonzero(s2)[near]
    D = np.linalg.norm(d1[s1][:, None] - d2[None], axis=2)  # against all K points of image 2
    res = {}
    for eps in EPS:
        ok = near_d <= eps
        res[f"rep@{eps}"] = ok.sum() / n
        rows = np.flatnonzero(ok)
        true = D[rows, idx2[ok]]
        other = D[rows].copy()
        other[np.arange(len(rows)), idx2[ok]] = np.inf
        rank = 1 + (other <= true[:, None]).sum(axis=1)
        res[f"nn@{eps}"] = np.mean(rank == 1)
        best = other.min(axis=1)
        with np.errstate(divide="ignore", invalid="ignore"):
            ratio = np.where((true == 0) & (best == 0), 1.0, true / best)  # 0/0 is a tie
        res[f"ratio@{eps}"] = np.median(ratio)
    res["n"] = n
    return res


def warp(img, H):
    return transform.warp(img, H.inverse, order=1, mode="constant", cval=0.0, preserve_range=True)


if __name__ == "__main__":
    tex, cam = texture(), camera()
    print("texture periodic: max|img - shift by P| =",
          max(np.abs(tex - np.roll(tex, P, 0)).max(), np.abs(tex - np.roll(tex, P, 1)).max()))
    rng = np.random.default_rng(123)
    Fs = np.maximum(rng.normal(size=(20, 20, 6)), 0)
    keep, _ = d2_detect(Fs)
    print("D2 rule vectorized == brute force:", bool(np.array_equal(keep, d2_bruteforce(Fs))))
    src = np.array([[0, 0], [N - 1, 0], [N - 1, N - 1], [0, N - 1]], float)

    def random_h(seed):
        H = transform.ProjectiveTransform()
        H.estimate(src, src + np.random.default_rng(seed).uniform(-JITTER * N, JITTER * N, size=(4, 2)))
        return H

    yy, xx = np.mgrid[0:N, 0:N]
    errs = []
    for seed in SEEDS:
        H = random_h(seed)
        for x0 in (128.0, 256.0, 384.0):
            for y0 in (128.0, 256.0, 384.0):
                wd = warp(np.exp(-((xx - x0) ** 2 + (yy - y0) ** 2) / 8.0), H)
                px, py = H(np.array([[x0, y0]]))[0]
                win = (np.abs(xx - px) <= 8) & (np.abs(yy - py) <= 8)
                errs.append(np.hypot((wd * xx * win).sum() / (wd * win).sum() - px,
                                     (wd * yy * win).sum() / (wd * win).sum() - py))
    print(f"dot check (5 homographies x 9 dots): max |centroid - H(p)| = {max(errs):.4f} px")

    I = transform.ProjectiveTransform()
    print("\nBase case: identity homography, no noise")
    for name, img in (("texture", tex), ("camera", cam)):
        A, B = detectors(img), detectors(warp(img, I))
        for det in A:
            r = evaluate(*A[det], *B[det], I)
            print(f"  {name:<8}{det:<15} rep@1.5={r['rep@1.5']:.4f} rep@3={r['rep@3.0']:.4f} "
                  f"nn@3={r['nn@3.0']:.4f}")

    print("\nRandom homographies (seeds 0-4), noise sd 0.01: mean [min, max]")
    keys = ("n", "rep@1.5", "rep@3.0", "nn@1.5", "nn@3.0", "ratio@3.0")
    print(f"{'image':<8}{'detector':<15}" + "".join(f"{k:>22}" for k in keys))
    for name, img in (("texture", tex), ("camera", cam)):
        runs = []
        for seed in SEEDS:
            H = random_h(seed)
            g = np.random.default_rng(100 + seed)
            im1 = img + g.normal(0, NOISE, img.shape)
            im2 = warp(img, H) + g.normal(0, NOISE, img.shape)
            A, B = detectors(im1), detectors(im2)
            runs.append({det: evaluate(*A[det], *B[det], H) for det in A})
        for det in runs[0]:
            cells = []
            for k in keys:
                v = np.array([r[det][k] for r in runs], float)
                f = "{:.0f}" if k == "n" else "{:.3f}"
                cells.append((f + " [" + f + ", " + f + "]").format(v.mean(), v.min(), v.max()))
            print(f"{name:<8}{det:<15}" + "".join(f"{c:>22}" for c in cells))

Output:

texture periodic: max|img - shift by P| = 0.0
D2 rule vectorized == brute force: True
dot check (5 homographies x 9 dots): max |centroid - H(p)| = 0.0356 px

Base case: identity homography, no noise
  texture Harris          rep@1.5=1.0000 rep@3=1.0000 nn@3=0.0033
  texture D2 Eq. (3)      rep@1.5=1.0000 rep@3=1.0000 nn@3=0.0000
  texture D2 + edge test  rep@1.5=1.0000 rep@3=1.0000 nn@3=0.0000
  camera  Harris          rep@1.5=1.0000 rep@3=1.0000 nn@3=1.0000
  camera  D2 Eq. (3)      rep@1.5=1.0000 rep@3=1.0000 nn@3=1.0000
  camera  D2 + edge test  rep@1.5=1.0000 rep@3=1.0000 nn@3=1.0000

Random homographies (seeds 0-4), noise sd 0.01: mean [min, max]
image   detector                            n               rep@1.5               rep@3.0                nn@1.5                nn@3.0             ratio@3.0
texture Harris                 271 [263, 278]  0.369 [0.349, 0.384]  0.372 [0.349, 0.392]  0.010 [0.000, 0.029]  0.010 [0.000, 0.029]  4.884 [3.701, 5.895]
texture D2 Eq. (3)             273 [243, 290]  0.363 [0.330, 0.380]  0.372 [0.355, 0.387]  0.002 [0.000, 0.010]  0.002 [0.000, 0.009]  5.932 [4.200, 6.582]
texture D2 + edge test         270 [240, 290]  0.291 [0.196, 0.358]  0.310 [0.255, 0.358]  0.006 [0.000, 0.019]  0.005 [0.000, 0.014]  5.295 [2.910, 9.178]
camera  Harris                 279 [269, 287]  0.847 [0.807, 0.861]  0.888 [0.867, 0.907]  0.964 [0.952, 0.975]  0.950 [0.930, 0.975]  0.352 [0.319, 0.395]
camera  D2 Eq. (3)             296 [294, 298]  0.801 [0.736, 0.855]  0.970 [0.946, 0.990]  0.690 [0.601, 0.748]  0.654 [0.574, 0.701]  0.762 [0.698, 0.848]
camera  D2 + edge test         280 [268, 292]  0.785 [0.757, 0.835]  0.878 [0.846, 0.930]  0.993 [0.987, 1.000]  0.912 [0.892, 0.939]  0.304 [0.292, 0.324]

The script uses NumPy 2.3.5, SciPy 1.16.3 and scikit-image 0.25.2 with BLAS limited to one thread, and runs in about 15 seconds; nothing in it is random beyond the fixed seeds, and two runs printed identical output.

All checks pass. The identity case already shows the texture’s problem without any warp or noise: every point is found again exactly, yet its true partner is almost never strictly the nearest descriptor, because each periodic copy of a structure produces the same descriptor and ties count as failures.

H1 held. Under the homographies, the share of repeatable texture points whose partner is the nearest descriptor stays below 0.03 for every detector and every pair, at 3 pixels and at 1.5 pixels (its largest value is 0.029, for Harris). The median ratio of the partner’s distance to the nearest other point’s is about 5: for a typical repeatable texture point, some other point in image 2, in our reading another copy of the same structure, has a descriptor several times closer than the true partner’s. Nearest-neighbour matching, or a ratio test, would reject or mismatch nearly all of them.

H2 held. On camera the same descriptor makes the partner the nearest neighbour for 0.950 of Harris’s repeatable points, 0.654 of D2 Eq. (3)’s and 0.912 of the edge-tested D2 points’, against 0.010, 0.002 and 0.005 on the texture, and the gap is at least as large at 1.5 pixels. The descriptor and the matching rule are identical in both images; only the image content changed.

H3 failed. The texture points are much less repeatable than camera’s, not about as repeatable. At 3 pixels, the means are 0.372 [0.349, 0.392] for Harris, 0.372 [0.355, 0.387] for D2 Eq. (3) and 0.310 [0.255, 0.358] for the edge-tested variant, against 0.888 [0.867, 0.907], 0.970 [0.946, 0.990] and 0.878 [0.846, 0.930] on camera. At 1.5 pixels the texture gives 0.369 [0.349, 0.384], 0.363 [0.330, 0.380] and 0.291 [0.196, 0.358], against 0.847 [0.807, 0.861], 0.801 [0.736, 0.855] and 0.785 [0.757, 0.835]. The texture holds $32^2 = 1{,}024$ copies of its cell, and $464 / 16 = 29$ periods fit along each side of the region inside the 24-pixel margin, so each local maximum of a detector’s response recurs about $29^2 = 841$ times there (the filters reach past the margin, so copies near it can differ slightly), far more than the 300 points each detector keeps, so the top-300 selection must choose among many maxima of nearly equal score; in our reading, noise and the slight differences the warp introduces between copies decide which copies are kept in each image, so the two selections overlap only partly. The script does not test this explanation. The point the example set out to show survives in a weaker form: the texture points that are found again are almost all ambiguous, but on this texture repeatability itself was also low under this top-$K$ protocol.

H4 failed. On camera, D2 Eq. (3) gave nn@3 of 0.654 [0.574, 0.701] against Harris’s 0.950 [0.930, 0.975], and at 1.5 pixels 0.690 [0.601, 0.748] against 0.964 [0.952, 0.975]; every pair favoured Harris. With fixed filters, choosing points where a channel is locally maximal did not choose points as distinctive as Harris’s. D2 Eq. (3) had the highest repeatability at 3 pixels, 0.970, but dropped to 0.801 at 1.5 pixels, the largest drop of the three detectors.

The edge test, for which no outcome was predicted, changed the D2 rule’s results on camera substantially: nn@3 rose from 0.654 to 0.912 and the median ratio fell from 0.762 to 0.304, while repeatability at 3 pixels fell from 0.970 to 0.878. In our reading, Eq. (3) alone accepts maxima along edges, where a point can slide along the edge without changing its response much and where neighbouring points have similar descriptors; the edge test, which the official code adds and the paper does not mention, removes them. On the texture, the edge test lowered repeatability (0.310 against 0.372 at 3 pixels) and left the points ambiguous.

The example uses fixed filters, a single scale of detection and one descriptor for all detectors, so its numbers say nothing about how trained networks compare; what it shows is that the detector and the descriptor must be scored separately, and that a high repeatability score does not imply matchable points.

Common Misconceptions

• “A repeatable point is a good point.” Repeatability measures only whether a point is found again. On repetitive structure the point may be found again and still be matched to the wrong copy, as the example shows; the descriptor’s distinctiveness, and downstream matching or pose accuracy, have to be measured as well.
• “Learned features are invariant to everything.” They are invariant to what the training pairs varied. D2-Net’s authors attribute its weaker results on viewpoint changes to the small viewpoint changes in most of its training pairs.
• “Describe-and-detect gives better-localized points.” D2-Net’s authors report the opposite: its keypoints come from coarser, higher-level features and are less accurately localized, which showed at strict matching thresholds and in larger reprojection errors in their reconstruction experiment (Table 1), while being more robust under day–night changes.
• “Learned features have made SIFT obsolete.” In the 2017 comparison that the SIFT paper article summarizes, advanced hand-crafted variants performed on par with or better than the learned descriptors tested for reconstruction; SIFT remains COLMAP’s default extractor, as of November 2026.
• “A paper’s equations are the whole method.” D2-Net’s hard detection in its official code includes an edge test that the paper’s Eq. (3) does not state, and in the example that test changed the result substantially.

Where It Is Used

• Structure from motion, where learned extractors such as ALIKED can replace SIFT in COLMAP.
• Visual localization under day–night change and in indoor scenes, the two settings in which D2-Net’s authors reported state-of-the-art results for sparse features.
• Feature matching pipelines, where learned keypoints and descriptors feed nearest-neighbour matching or learned matchers, followed by geometric verification with RANSAC.