Evaluation & Metrics

Mean Per Joint Position Error

The average Euclidean distance between predicted and true 3D joints, and the family built on it; absolute and root-relative error, the choice of root and joint set, scale-normalized N-MPJPE, Procrustes-aligned PA-MPJPE, root position error and per-vertex error, with what each alignment removes and what it hides.

intermediate

The mean per joint position error (MPJPE) is the average straight-line distance, usually in millimetres, between the joints a 3D pose estimator predicts and the joints of the ground-truth skeleton. It is the headline number of 3D human pose estimation and of 3D hand pose. The formula is one line, but the number a paper reports depends on what is done to the prediction before the distances are taken: nothing (absolute error), a translation that puts the two roots together (root-relative error), a single scale factor (N-MPJPE), or a full Procrustes alignment (PA-MPJPE). It also depends on which joint is the root, which joints are averaged, and how frames are pooled. Each alignment removes a kind of error. It can also spread one joint’s error over the whole skeleton, or hide errors that a skeleton cannot represent. This article covers the family as a whole. The benchmark protocols built on it are summarized in 3D Human Pose Estimation and Human3.6M, and the alignment algorithm in the Kabsch algorithm.

Formula

For one person in one frame, with JJ predicted joints X^j\hat{\mathbf{X}}_j and true joints Xj\mathbf{X}_j in R3\mathbb{R}^3, and a transformation T\mathcal{T} applied to the prediction,

E(T)=1J∑j=1J∥T(X^j)−Xj∥2.E(\mathcal{T}) = \frac{1}{J} \sum_{j=1}^{J} \left\lVert \mathcal{T}(\hat{\mathbf{X}}_j) - \mathbf{X}_j \right\rVert_2 .

Ionescu et al. (2014, Eq. 8) define MPJPE for a frame, with the root at the origin of the prediction’s coordinate system, and the error of a set of frames as the mean over frames; Human3.6M gives the definition and how published numbers are aggregated. The members of the family differ only in T\mathcal{T}:

Name (as printed) T\mathcal{T} Removes Source
absolute MPJPE (descriptive name); 3DPCKabs_{abs}, “3DPCK without root alignment”, and MRPE for the root alone (Moon et al.) identity nothing Moon et al. (2019)
MPJPE, protocol #1 translation putting the predicted root on the true root the person’s position Martinez et al. (2017)
NMPJPE (Rhodin et al.), N-MPJPE (Pavllo et al.) root alignment, then one scale ss position and overall size Rhodin et al. (2018); Pavllo et al. (2019)
PMPJPE, P-MPJPE, PA-MPJPE, reconstruction error least-squares rotation and translation, with or without scale position, orientation and size see 3D Human Pose Estimation

Root-relative error. With root joint rr, T(X^)=X^−X^r+Xr\mathcal{T}(\hat{\mathbf{X}}) = \hat{\mathbf{X}} - \hat{\mathbf{X}}_r + \mathbf{X}_r, so the error compares X^j−X^r\hat{\mathbf{X}}_j - \hat{\mathbf{X}}_r with Xj−Xr\mathbf{X}_j - \mathbf{X}_r. Martinez et al. (2017) describe the standard Human3.6M evaluation as the average error over all joints and cameras after aligning the root, “central hip”, joint, and call it protocol #1.

N-MPJPE. Rhodin et al. (2018) make the error “independent from the subjects’ height” by applying a single scale factor to the prediction so as to minimize the squared distance to the label; their poses are relative to the pelvis, and orientation is left unchanged. They call the result NMPJPE, and NPCK for the percentage of correct keypoints within 15 cm. With root-relative prediction pj\mathbf{p}_j and truth gj\mathbf{g}_j, the least-squares scale has a closed form (our derivation; the paper states the criterion in words):

s∗=arg⁡min⁡s∑j∥s pj−gj∥2=∑jpj⊤gj∑j∥pj∥2,EN-MPJPE=1J∑j∥s∗pj−gj∥2.s^\ast = \arg\min_s \sum_j \lVert s\,\mathbf{p}_j - \mathbf{g}_j \rVert^2 = \frac{\sum_j \mathbf{p}_j^\top \mathbf{g}_j}{\sum_j \lVert \mathbf{p}_j \rVert^2}, \qquad E_{\text{N-MPJPE}} = \frac{1}{J}\sum_j \lVert s^\ast \mathbf{p}_j - \mathbf{g}_j \rVert_2 .

Pavllo et al. (2019) list it as protocol 3, which “aligns predicted poses with the ground-truth only in scale (N-MPJPE)”, and use it for their semi-supervised experiments. The work they say they follow is their reference 45, a different 2018 paper by Rhodin, Salzmann and Fua (ECCV, “Unsupervised geometry-aware representation for 3D human pose estimation”, not read), not the CVPR paper of Rhodin et al. described above.

PA-MPJPE. The prediction is aligned by the rotation and translation, and in some implementations the scale, that minimize the summed squared joint distances, the problem the Kabsch algorithm and Umeyama’s extension solve; 3D Human Pose Estimation describes the reference implementation and the papers that leave the scale out.

Root position error. Moon et al. (2019, Eq. 5) introduce the mean root position error, MRPE =1N∑i∥R(i)−R(i)∗∥2= \frac{1}{N}\sum_i \lVert \mathbf{R}^{(i)} - \mathbf{R}^{(i)\ast} \rVert_2, the mean over NN test samples of the distance between the estimated and true root coordinates, to evaluate absolute localization separately from root-relative pose.

Per-vertex error. Mesh methods replace joints by surface vertices of a common mesh. Pavlakos et al. (2018) report the “mean per-vertex error” between predicted and ground-truth shape on UP-3D and SURREAL. The paper does not print its formula; read as its name says, it is the same mean of distances over the vertices of the SMPL mesh.

All members have the units of the coordinates: millimetres in 3D, pixels for 2D joints.

Intuition

MPJPE asks how far, on average, each joint is from where it should be. Each alignment first moves the prediction as close to the truth as a chosen class of transformations allows, and then asks the same question. The larger the class, the more kinds of error are forgiven:

  • No alignment charges everything, including where the person stands and how far from the camera, a distance that a single image does not record.
  • Root alignment forgives the person’s position, but it treats the root joint as exact. Any error at the root is copied onto every other joint.
  • Scale alignment also forgives the overall size of the person. A tall person far away and a short person near the camera can project to nearly the same image.
  • Procrustes alignment also forgives orientation. What is left is the shape of the skeleton.

What none of them sees is anything a joint position cannot represent: the twist of a limb about its own axis, the width of the body, the surface. The per-vertex error measures those.

Interpretation

  • Range. [0,∞)[0, \infty); 0 means the aligned skeletons coincide. There is no upper bound and no normalization by body size, except in N-MPJPE and PA-MPJPE with scale.
  • Units. Millimetres for 3D results on body benchmarks. A 50 mm MPJPE means the joints are, on average, 5 cm from the truth after the stated alignment. It does not mean every joint is within 5 cm. As a mean of distances, not squared distances, it is less dominated by one bad joint than a squared error such as mean squared error or its root, but it still grows linearly with that joint’s error (Human3.6M discusses the perceptual MPJLE that Ionescu et al. proposed for this reason).
  • Comparability. Two MPJPE values are comparable only with the same alignment, root, joint set, frame sampling, camera set and aggregation. Martinez et al.’s Table 3 lists methods evaluated with 14 and with 17 joints side by side, marked “14j” and “17j”. Moon et al. (2019) number their Human3.6M protocols the other way round from Martinez et al.: their “Protocol 1” trains on six subjects, tests on S11 and reports PA MPJPE, and their “Protocol 2” tests on S9 and S11 with MPJPE after root alignment. 3D Human Pose Estimation collects further differences in what “Protocol #2” means.
  • Ordering. In Rhodin et al.’s Table 1, every method with all three values has MPJPE above NMPJPE above PMPJPE. The ordering is not guaranteed. Each alignment minimizes a sum of squared distances, while MPJPE averages unsquared distances, so an alignment can raise MPJPE (see Failure Cases and the example).

When to Use It

  • Root-relative MPJPE for monocular single-person methods that do not estimate the person’s distance from the camera, the setting in which Human3.6M results are usually reported.
  • Absolute MPJPE, MRPE or 3DPCKabs_{abs} when the position of the person matters: multi-person scenes, multi-view systems, and any downstream use in a real coordinate frame. Moon et al. (2019) introduce MRPE to evaluate the localization of the absolute root and report it beside root-relative MPJPE (their Table 1). According to 3D Human Pose Estimation, Iskakov et al. (2019) excluded scenes of one Human3.6M subject whose ground truth is shifted in 3D, an error that subtracting the pelvis hides, before reporting absolute error.
  • N-MPJPE when the method cannot know the subject’s metric size. Rhodin et al. trained some networks with a scale-normalized pose distance (their NSE, Eq. 4, which divides each pose by its norm), and their Table 1 marks MPJPE “not applicable” for those networks while giving NMPJPE and PMPJPE.
  • PA-MPJPE to compare articulated shape independently of global orientation, for instance across datasets whose camera conventions differ, keeping in mind that two variants (with and without scale) are in use.
  • Per-vertex error when the output is a body mesh and the surface matters: clothing, shape, contact, avatars.

Papers report several at once: Rhodin et al.’s Table 1 has MPJPE, NMPJPE and PMPJPE columns; their supervised baseline trained with the plain squared error scores 66.8, 63.3 and 51.6 mm on Human3.6M (S9 and S11, 10 fps), in that order.

Failure Cases

  • A root error becomes every joint’s error. Root alignment subtracts the predicted root from every predicted joint. If only the root is wrong, by δ\boldsymbol\delta, the absolute error is ∥δ∥/J\lVert\boldsymbol\delta\rVert / J while the root-relative error is (J−1)∥δ∥/J(J-1)\lVert\boldsymbol\delta\rVert / J for JJ joints, the root included with zero error: 16 times larger for 17 joints. A root defined from other joints, such as the midpoint of the hips, is unaffected by an error at the pelvis joint, though an error in one hip still moves it by half that error.
  • Alignment spreads one joint’s error. If one wrist is wrong and everything else is right, root alignment leaves the error on the wrist. A least-squares alignment moves the whole skeleton toward the wrong wrist, giving every joint a small error and the wrist a slightly smaller one. Because MPJPE averages unsquared distances, the total can rise. The example’s single-wrist case has PA-MPJPE about twice the root-relative error.
  • Scale normalization is not free either. The least-squares scale lowers the squared error but can raise MPJPE when the error is concentrated in some joints, as in the example’s limb-length bias.
  • Twist and surface are invisible. Joint positions do not record a limb’s rotation about its own axis, and a skeleton has no width. A forearm turned by 180° about its axis, or a body 10% wider, has zero MPJPE as long as its joints stay where they were, as in the example’s tube; the per-vertex error sees both. Kanazawa et al. (2018) observe, as 3D Human Pose Estimation reports, that MPJPE does not correlate well with the visual quality of results.
  • Alignment of the evaluation, not of the method. PA-MPJPE uses the ground truth to align each prediction, information no deployed system has. A method can score well in PA-MPJPE with a wrong orientation and scale.
  • Hidden conventions. Whether the root is counted in the mean (Human3.6M shows that 3d-pose-baseline’s 17-joint mean with a zero root error is exactly 16/17 of the 16-joint mean), which joints are kept, and whether frames or actions are averaged all move the number; in Human3.6M’s example the root convention alone moves it by 2.6 mm.

Variants

  • Joint sets and roots. Pavllo et al. (2019) adopt a 17-joint skeleton on Human3.6M following previous work, and 3D Human Pose Estimation notes that the root is usually the pelvis; Martinez et al.’s Table 3 also lists 14-joint results. Rhodin et al. predict 16 joints relative to the pelvis. The root is the pelvis or central hip in these papers.
  • 2D MPJPE. The same mean over image coordinates, in pixels; Ionescu et al. state that in 2D the error is reported in pixels, and Pavllo et al. use a “2D MPJPE loss” on reprojected joints for their semi-supervised training.
  • Weighted MPJPE (WMPJPE). Pavllo et al. (2019, Eq. 1) weight the trajectory error of each sample by the inverse ground-truth depth, as a training loss for the global trajectory: regressing a precise trajectory for distant subjects is harder and, for back-projection, unnecessary.
  • MPJPE for hands. Keypoint Detection describes InterHand2.6M’s MPJPE after aligning each hand’s root, and its mean relative-root position error (MRRPE), the error of the left hand’s root relative to the right hand’s.
  • Per-vertex error (PVE) and its aligned form. Pavlakos et al. (2018) evaluate mesh recovery by mean per-vertex error; in their UP-3D ablation (Table 1), adding a per-vertex loss to the rotation-matrix parameter loss lowers it from 140.7 to 120.7 mm with Priors trained on UP-3D. They give the reason that the loss correlates better with the vertex-to-vertex error used for evaluation. Parametric Human Body Models reports SMPLify-X’s vertex-to-vertex error after Procrustes alignment, the per-vertex analogue of PA-MPJPE.
  • Velocity and acceleration errors, UMPJPE, MPJAE, MPJLE. Different measures with similar names: velocity and acceleration errors are described in 3D Human Pose Estimation, and the universal, angle and localization variants of Ionescu et al. in Human3.6M.

Example

The example builds predictions from known error components and prints what each version of the metric makes of them. Part A poses 200 random skeletons of 17 joints in Human3.6M’s joint order (bone lengths of 115 to 445 mm, random local rotations of the heading, spine, neck, hips, knees, shoulders and elbows). The pelvis is placed at x∈[−500,500]x \in [-500, 500], y=900y = 900, z∈[3500,5500]z \in [3500, 5500] mm in a right-handed frame with yy up; the frame matters only for the absolute error. Each prediction is the ground truth with one component of error, then with all of them:

  • T: the whole pose moved by (0,0,200)(0, 0, 200) mm, a depth error;
  • R0: the pelvis joint alone moved by 50 mm in a random direction, a root-localization error;
  • S: the pose scaled by 0.9 about the pelvis;
  • Rot: a rotation by 15° about a random axis through the pelvis;
  • A: articulation noise, a random rotation vector with 4° standard deviation per component multiplied onto the local rotation of every non-root joint that has a child (bone lengths unchanged);
  • L: a limb-length bias, thighs and shins 5% too long, upper arms and forearms 5% too short;
  • W: the right wrist alone moved by 100 mm in a random direction;
  • ALL: A and L in the kinematics, then S, Rot, T, R0 and W in that order.

Seven measures are printed as means over the 200 poses: absolute error (abs17); root-relative error with the pelvis as root (rr17p) and with the midpoint of the hips as root (rr17h); the mid-hip error on 14 joints without pelvis, spine and thorax (rr14h); N-MPJPE with Rhodin et al.’s least-squares scale on pelvis-relative poses (N17p); and PA-MPJPE with a rigid alignment (PArig) and with a similarity alignment (PAsim), both centred on the centroids with reflections excluded. The scale and alignment are checked against SciPy’s minimize_scalar and orthogonal_procrustes, and the script checks that each alignment’s sum of squared errors is not above the root-aligned one, which least squares guarantees.

Part B is a forearm-like tube of radius 40 mm and length 250 mm along xx, with 26 rings of 24 vertices; its two joints, elbow and wrist, are the centroids of the end rings, a stand-in for the joint regressor of a body model. It is twisted about its axis by 0 to 180°, either rigidly or by an angle growing linearly from the elbow to the wrist, and widened by w=1.1w = 1.1; the joint MPJPE, the per-vertex error (PVE), its difference from the closed form, and the PVE after rigid and similarity Procrustes alignment of the 624 vertices are printed. The script uses NumPy 2.3.5 and SciPy 1.16.3 (Python 3.13) and runs in about half a second.

import os
for v in ("OMP_NUM_THREADS", "OPENBLAS_NUM_THREADS", "MKL_NUM_THREADS", "VECLIB_MAXIMUM_THREADS"):
    os.environ[v] = "1"                                      # one BLAS thread, set before NumPy is imported
import numpy as np
from scipy.linalg import orthogonal_procrustes
from scipy.optimize import minimize_scalar
from scipy.spatial.transform import Rotation

# 17 joints in Human3.6M order; parent, rest direction (y up, person faces +z), length in mm
NAMES = ["pelvis", "r_hip", "r_knee", "r_ankle", "l_hip", "l_knee", "l_ankle", "spine", "thorax",
         "neck", "head", "l_shoulder", "l_elbow", "l_wrist", "r_shoulder", "r_elbow", "r_wrist"]
BONES = {1: (0, (-1, 0, 0), 135), 2: (1, (0, -1, 0), 445), 3: (2, (0, -1, 0), 435),
         4: (0, (1, 0, 0), 135), 5: (4, (0, -1, 0), 445), 6: (5, (0, -1, 0), 435),
         7: (0, (0, 1, 0), 235), 8: (7, (0, 1, 0), 245), 9: (8, (0, 1, 0), 115),
         10: (9, (0, 1, 0), 115), 11: (8, (1, 0, 0), 165), 12: (11, (0, -1, 0), 285),
         13: (12, (0, -1, 0), 255), 14: (8, (-1, 0, 0), 165), 15: (14, (0, -1, 0), 285),
         16: (15, (0, -1, 0), 255)}
LIMB = {2: 1.05, 3: 1.05, 5: 1.05, 6: 1.05, 12: 0.95, 13: 0.95, 15: 0.95, 16: 0.95}  # L: bone -> factor
HAS_CHILD = sorted({p for p, _, _ in BONES.values()} - {0})
J14 = [j for j in range(17) if j not in (0, 7, 8)]          # drop pelvis, spine, thorax
N_POSES = 200

def R_(axis, deg):
    return Rotation.from_euler(axis, deg, degrees=True).as_matrix()

def sample_angles(rng):
    """Local rotation of each joint (3x3) for one random pose."""
    L = {j: np.eye(3) for j in range(17)}
    L[0] = R_("yxz", [rng.uniform(-180, 180), rng.uniform(-10, 10), rng.uniform(-10, 10)])
    L[7] = R_("yxz", [rng.uniform(-30, 30), rng.uniform(-10, 30), rng.uniform(-15, 15)])
    L[9] = R_("yx", [rng.uniform(-45, 45), rng.uniform(-20, 30)])
    for hip, knee in ((1, 2), (4, 5)):
        L[hip] = R_("yxz", [rng.uniform(-20, 20), rng.uniform(-100, 30), rng.uniform(-20, 20)])
        L[knee] = R_("x", rng.uniform(0, 130))
    for sh, el, side in ((11, 12, 1), (14, 15, -1)):
        L[sh] = R_("yxz", [rng.uniform(-30, 30), rng.uniform(-70, 40), side * rng.uniform(0, 70)])
        L[el] = R_("x", -rng.uniform(0, 140))
    return L

def fk(L, root, limb=None):
    """Forward kinematics: joint positions (17, 3) in mm, pelvis at `root`."""
    X = np.zeros((17, 3)); X[0] = root; Rw = {0: L[0]}
    for j in range(1, 17):
        p, d, length = BONES[j]
        f = limb.get(j, 1.0) if limb else 1.0
        X[j] = X[p] + Rw[p] @ (f * length * np.array(d, float))
        Rw[j] = Rw[p] @ L[j]
    return X

def unit(rng):
    v = rng.normal(size=3); return v / np.linalg.norm(v)

# ---------- measures ----------
def mean_norm(P, G): return np.linalg.norm(P - G, axis=-1).mean()

def rel(X, root): return X - (X[0] if root == "pelvis" else 0.5 * (X[1] + X[4]))

def n_scale(p, g):
    """Rhodin et al.: one scale on the prediction minimizing the squared distance."""
    return (p * g).sum() / (p * p).sum()

def procrustes(p, g, scale):
    """Similarity (scale=True) or rigid alignment of p onto g; returns aligned p."""
    mp, mg = p.mean(0), g.mean(0)
    p0, g0 = p - mp, g - mg
    U, s, Vt = np.linalg.svd(p0.T @ g0)
    D = np.diag([1, 1, np.sign(np.linalg.det(U @ Vt))])      # no reflections
    Rm = U @ D @ Vt                                          # p0 @ Rm ~ g0
    a = (s * np.diag(D)).sum() / (p0 ** 2).sum() if scale else 1.0
    return a * p0 @ Rm + mg, Rm, a

def measures(P, G):
    pp, gp = rel(P, "pelvis"), rel(G, "pelvis")
    ph, gh = rel(P, "midhip"), rel(G, "midhip")
    return np.array([
        mean_norm(P, G),                                     # abs17
        mean_norm(pp, gp),                                   # rr17p
        mean_norm(ph, gh),                                   # rr17h
        mean_norm(ph[J14], gh[J14]),                         # rr14h
        mean_norm(n_scale(pp, gp) * pp, gp),                 # N17p
        mean_norm(procrustes(P, G, False)[0], G),            # PArig17
        mean_norm(procrustes(P, G, True)[0], G)])            # PAsim17
COLS = ["abs17", "rr17p", "rr17h", "rr14h", "N17p", "PArig", "PAsim"]

# ---------- data ----------
rng = np.random.default_rng(0)
poses = []
for i in range(N_POSES):
    L = sample_angles(rng)
    root = np.array([rng.uniform(-500, 500), 900.0, rng.uniform(3500, 5500)])
    poses.append(dict(L=L, root=root, G=fk(L, root),
                      artic={j: Rotation.from_rotvec(np.radians(4) * rng.normal(size=3)).as_matrix()
                             for j in HAS_CHILD},
                      d_root=50 * unit(rng), d_wrist=100 * unit(rng),
                      rot=Rotation.from_rotvec(np.radians(15) * unit(rng)).as_matrix()))
T = np.array([0.0, 0.0, 200.0])

def predict(ps, comps):
    L, root, G = ps["L"], ps["root"], ps["G"]
    if "A" in comps or "L" in comps:
        La = {j: (L[j] @ ps["artic"][j] if ("A" in comps and j in ps["artic"]) else L[j]) for j in L}
        P = fk(La, root, LIMB if "L" in comps else None)
    else:
        P = G.copy()
    if "S" in comps: P = root + 0.9 * (P - root)
    if "Rot" in comps: P = root + (P - root) @ ps["rot"].T
    if "T" in comps: P = P + T
    if "R0" in comps: P[0] += ps["d_root"]
    if "W" in comps: P[16] += ps["d_wrist"]
    return P

CONDS = [("T", {"T"}), ("R0", {"R0"}), ("S", {"S"}), ("Rot", {"Rot"}), ("A", {"A"}),
         ("L", {"L"}), ("W", {"W"}), ("ALL", {"A", "L", "S", "Rot", "T", "R0", "W"})]

# ---------- checks on the data and the implementations ----------
blen = np.array([[np.linalg.norm(p["G"][j] - p["G"][BONES[j][0]]) - BONES[j][2] for j in BONES] for p in poses])
mid = max(np.linalg.norm(rel(p["G"], "midhip")[0]) for p in poses)
PA_ = [predict(p, {"A"}) for p in poses]
blenA = max(abs(np.linalg.norm(P[j] - P[BONES[j][0]]) - BONES[j][2]) for P in PA_ for j in BONES)
PL_ = [predict(p, {"L"}) for p in poses]
blenL = max(abs(np.linalg.norm(P[j] - P[BONES[j][0]]) - LIMB.get(j, 1.0) * BONES[j][2]) for P in PL_ for j in BONES)
print("data checks")
print(f"  {N_POSES} poses x 17 joints; ground-truth bone length error {np.abs(blen).max():.1e} mm, "
      f"pelvis - mid-hip {mid:.1e} mm")
print(f"  A keeps bone lengths: {blenA:.1e} mm; L scales them as planned: {blenL:.1e} mm")

g_n = g_s = 0.0; d_rot = 0.0; n_rot = 0; sse_viol = np.inf
for p in poses[:50]:
    P, G = predict(p, {"A", "L", "S", "Rot", "R0"}), p["G"]
    pp, gp = rel(P, "pelvis"), rel(G, "pelvis")
    s_opt = minimize_scalar(lambda s: ((s * pp - gp) ** 2).sum(), bracket=(0.5, 1.5), tol=1e-12).x
    g_n = max(g_n, abs(n_scale(pp, gp) - s_opt))
    Al, Rm, a = procrustes(P, G, True)
    a_opt = minimize_scalar(lambda s: ((s * (P - P.mean(0)) @ Rm - (G - G.mean(0))) ** 2).sum(),
                            bracket=(0.5, 1.5), tol=1e-12).x
    g_s = max(g_s, abs(a - a_opt))
    Rs = orthogonal_procrustes(P - P.mean(0), G - G.mean(0))[0]
    if np.linalg.det(Rs) > 0:                                # scipy allows reflections; compare proper rotations
        d_rot = max(d_rot, np.abs(Rs - Rm).max()); n_rot += 1
    sse = lambda A, B: ((A - B) ** 2).sum()
    sse_viol = min(sse_viol, sse(pp, gp) - sse(Al, G), sse(pp, gp) - sse(procrustes(P, G, False)[0], G),
                   sse(pp, gp) - sse(n_scale(pp, gp) * pp, gp))
print("implementation checks (50 poses, condition A+L+S+Rot+R0)")
print(f"  N scale vs minimize_scalar {g_n:.1e}; PA scale vs minimize_scalar {g_s:.1e}; "
      f"PA rotation vs scipy orthogonal_procrustes {d_rot:.1e} ({n_rot} of 50 poses compared)")
print(f"  smallest SSE(root-aligned) - SSE(aligned) over N, PA rigid, PA sim: {sse_viol:.3e} mm^2 (>= 0 expected)")

# ---------- part A: what each alignment removes ----------
res = {}
for name, comps in CONDS:
    res[name] = np.mean([measures(predict(p, comps), p["G"]) for p in poses], axis=0)
print("\nA. mean over 200 poses (mm)")
print(f"  {'cond':4s} " + " ".join(f"{c:>9s}" for c in COLS))
for name, _ in CONDS:
    print(f"  {name:4s} " + " ".join(f"{v:9.4f}" for v in res[name]))

g0 = np.mean([np.linalg.norm(rel(p["G"], "pelvis"), axis=1).mean() for p in poses])
print("\nA. identities (difference from the derived value, mm)")
ident = [("T abs - 200", res["T"][0] - 200), ("T max(rr17p..PAsim)", res["T"][1:].max()),
         ("R0 abs - 50/17", res["R0"][0] - 50 / 17), ("R0 rr17p - 800/17", res["R0"][1] - 800 / 17),
         ("R0 rr17h - 50/17", res["R0"][2] - 50 / 17), ("R0 rr14h", res["R0"][3]),
         ("S abs - rr17p", res["S"][0] - res["S"][1]), ("S abs - 0.1*mean|g-g0|", res["S"][0] - 0.1 * g0),
         ("S max(N, PAsim)", max(res["S"][4], res["S"][6])),
         ("Rot abs - rr17p", res["Rot"][0] - res["Rot"][1]), ("Rot max(PArig, PAsim)", res["Rot"][5:].max()),
         ("W abs - 100/17", res["W"][0] - 100 / 17), ("W rr17p - 100/17", res["W"][1] - 100 / 17),
         ("W rr17h - 100/17", res["W"][2] - 100 / 17), ("W rr14h - 100/14", res["W"][3] - 100 / 14),
         ("L rr14h - 17/14*rr17h", res["L"][3] - 17 / 14 * res["L"][2])]
for k, v in ident:
    print(f"  {k:24s} {v: .1e}  {'held' if abs(v) <= 1e-9 else 'FAILED'}")

print("\nA. comparisons (mm)")
r = res
cmp_ = [("H2 R0: abs < PAsim < rr17p", f"{r['R0'][0]:.4f} < {r['R0'][6]:.4f} < {r['R0'][1]:.4f}",
         r["R0"][0] < r["R0"][6] < r["R0"][1]),
        ("H3 W: PArig > rr17p", f"{r['W'][5]:.4f} > {r['W'][1]:.4f}", r["W"][5] > r["W"][1]),
        ("H3 W: PAsim > rr17p", f"{r['W'][6]:.4f} > {r['W'][1]:.4f}", r["W"][6] > r["W"][1])]
for c in ("Rot", "A", "L", "ALL"):
    cmp_.append((f"H4 {c}: N17p < rr17p", f"{r[c][4]:.4f} < {r[c][1]:.4f}", r[c][4] < r[c][1]))
for c, _ in CONDS:                                           # equal within 1e-9 mm counts as <=
    d = r[c][6] - r[c][5]
    cmp_.append((f"H5 {c}: PAsim <= PArig", f"PAsim - PArig = {d: .1e}", d <= 1e-9))
cmp_ += [("H6 A: rr14h > rr17h", f"{r['A'][3]:.4f} > {r['A'][2]:.4f}", r["A"][3] > r["A"][2]),
         ("H7 ALL: abs > rr17p > PAsim", f"{r['ALL'][0]:.4f} > {r['ALL'][1]:.4f} > {r['ALL'][6]:.4f}",
          r["ALL"][0] > r["ALL"][1] > r["ALL"][6])]
for k, s, ok in cmp_:
    print(f"  {k:28s} {s:32s} {'true' if ok else 'false'}")

# ---------- part B: what joints cannot see ----------
RAD, LEN, NR, NV = 40.0, 250.0, 26, 24                       # forearm tube, mm; rings x vertices per ring
xk = np.linspace(0, LEN, NR)
th = 2 * np.pi * np.arange(NV) / NV
V = np.stack([np.repeat(xk, NV), RAD * np.tile(np.cos(th), NR), RAD * np.tile(np.sin(th), NR)], 1)
def joints(V):                                               # elbow, wrist = centroids of the end rings
    return np.stack([V[:NV].mean(0), V[-NV:].mean(0)])
Jt = joints(V)

def deform(V, phi, w, graded):
    ang = np.radians(phi) * (V[:, 0] / LEN if graded else 1.0)
    c, s = np.cos(ang), np.sin(ang)
    y, z = w * V[:, 1], w * V[:, 2]
    return np.stack([V[:, 0], c * y - s * z, s * y + c * z], 1)

print("\nB. forearm tube (r = 40 mm, 250 mm, 26 x 24 vertices): joint MPJPE and per-vertex error (mm)")
print(f"  {'twist':>5s} {'w':>4s} {'mode':>6s} {'MPJPE':>8s} {'PVE':>8s} {'|PVE-an|':>8s} {'PA-PVE rig':>10s} {'PA-PVE sim':>10s}")
pa_rig_max = 0.0
for graded in (False, True):
    for phi in (0, 30, 90, 180):
        for w in (1.0, 1.1):
            Vd = deform(V, phi, w, graded)
            Jd = joints(Vd)
            ang = np.radians(phi) * (xk / LEN if graded else np.ones(NR))
            an = np.mean(RAD * np.sqrt(w * w + 1 - 2 * w * np.cos(ang)))
            pve = mean_norm(Vd, V)
            print(f"  {phi:5d} {w:4.1f} {'graded' if graded else 'rigid':>6s} {mean_norm(Jd, Jt):8.1e} {pve:8.4f} "
                  f"{abs(pve - an):8.1e} {mean_norm(procrustes(Vd, V, False)[0], V):10.4f} "
                  f"{mean_norm(procrustes(Vd, V, True)[0], V):10.4f}")
            if not graded and w == 1.0:
                pa_rig_max = max(pa_rig_max, mean_norm(procrustes(Vd, V, False)[0], V))
print(f"  largest PA-PVE rig over rigid twists at w = 1: {pa_rig_max:.1e} mm")
data checks
  200 poses x 17 joints; ground-truth bone length error 5.7e-13 mm, pelvis - mid-hip 4.5e-13 mm
  A keeps bone lengths: 6.0e-13 mm; L scales them as planned: 5.7e-13 mm
implementation checks (50 poses, condition A+L+S+Rot+R0)
  N scale vs minimize_scalar 4.8e-09; PA scale vs minimize_scalar 1.3e-09; PA rotation vs scipy orthogonal_procrustes 0.0e+00 (50 of 50 poses compared)
  smallest SSE(root-aligned) - SSE(aligned) over N, PA rigid, PA sim: 8.841e+01 mm^2 (>= 0 expected)

A. mean over 200 poses (mm)
  cond     abs17     rr17p     rr17h     rr14h      N17p     PArig     PAsim
  T     200.0000    0.0000    0.0000    0.0000    0.0000    0.0000    0.0000
  R0      2.9412   47.0588    2.9412    0.0000   45.2496    5.6073    5.6353
  S      44.4459   44.4459   44.4459   48.9023    0.0000   41.5644    0.0000
  Rot    92.2599   92.2599   92.2599  101.4012   91.8045    0.0000    0.0000
  A      37.6273   37.6273   37.6273   44.1559   37.8708   31.0476   30.9734
  L      10.8380   10.8380   10.8380   13.1604   12.2727   12.5695   11.4928
  W       5.8824    5.8824    5.8824    7.1429    8.4512   12.1548   12.6039
  ALL   225.1800  116.7046  109.6868  118.0628  113.9411   54.0453   39.8025

A. identities (difference from the derived value, mm)
  T abs - 200               0.0e+00  held
  T max(rr17p..PAsim)       3.8e-13  held
  R0 abs - 50/17            2.2e-15  held
  R0 rr17p - 800/17         4.3e-14  held
  R0 rr17h - 50/17          2.2e-15  held
  R0 rr14h                  0.0e+00  held
  S abs - rr17p             0.0e+00  held
  S abs - 0.1*mean|g-g0|    0.0e+00  held
  S max(N, PAsim)           8.6e-13  held
  Rot abs - rr17p           0.0e+00  held
  Rot max(PArig, PAsim)     8.6e-13  held
  W abs - 100/17            8.9e-16  held
  W rr17p - 100/17          8.9e-16  held
  W rr17h - 100/17          8.9e-16  held
  W rr14h - 100/14         -1.8e-14  held
  L rr14h - 17/14*rr17h     7.1e-15  held

A. comparisons (mm)
  H2 R0: abs < PAsim < rr17p   2.9412 < 5.6353 < 47.0588        true
  H3 W: PArig > rr17p          12.1548 > 5.8824                 true
  H3 W: PAsim > rr17p          12.6039 > 5.8824                 true
  H4 Rot: N17p < rr17p         91.8045 < 92.2599                true
  H4 A: N17p < rr17p           37.8708 < 37.6273                false
  H4 L: N17p < rr17p           12.2727 < 10.8380                false
  H4 ALL: N17p < rr17p         113.9411 < 116.7046              true
  H5 T: PAsim <= PArig         PAsim - PArig =  2.4e-14         true
  H5 R0: PAsim <= PArig        PAsim - PArig =  2.8e-02         false
  H5 S: PAsim <= PArig         PAsim - PArig = -4.2e+01         true
  H5 Rot: PAsim <= PArig       PAsim - PArig =  1.6e-14         true
  H5 A: PAsim <= PArig         PAsim - PArig = -7.4e-02         true
  H5 L: PAsim <= PArig         PAsim - PArig = -1.1e+00         true
  H5 W: PAsim <= PArig         PAsim - PArig =  4.5e-01         false
  H5 ALL: PAsim <= PArig       PAsim - PArig = -1.4e+01         true
  H6 A: rr14h > rr17h          44.1559 > 37.6273                true
  H7 ALL: abs > rr17p > PAsim  225.1800 > 116.7046 > 39.8025    true

B. forearm tube (r = 40 mm, 250 mm, 26 x 24 vertices): joint MPJPE and per-vertex error (mm)
  twist    w   mode    MPJPE      PVE |PVE-an| PA-PVE rig PA-PVE sim
      0  1.0  rigid  0.0e+00   0.0000  0.0e+00     0.0000     0.0000
      0  1.1  rigid  1.7e-15   4.0000  4.5e-14     4.0000     3.4285
     30  1.0  rigid  3.2e-15  20.7055  3.6e-15     0.0000     0.0000
     30  1.1  rigid  3.3e-15  22.0815  1.1e-14     4.0000     3.4285
     90  1.0  rigid  6.5e-15  56.5685  7.1e-15     0.0000     0.0000
     90  1.1  rigid  4.0e-15  59.4643  1.4e-14     4.0000     3.4285
    180  1.0  rigid  6.6e-15  80.0000  0.0e+00     0.0000     0.0000
    180  1.1  rigid  4.6e-15  84.0000  0.0e+00     4.0000     3.4285
      0  1.0 graded  0.0e+00   0.0000  0.0e+00     0.0000     0.0000
      0  1.1 graded  1.7e-15   4.0000  4.5e-14     4.0000     3.4285
     30  1.0 graded  1.6e-15  10.4099  3.6e-15     5.4370     5.4350
     30  1.1 graded  2.5e-15  12.0223  2.7e-14     7.2816     6.7792
     90  1.0 graded  3.2e-15  29.7719  0.0e+00    16.1111    16.0533
     90  1.1 graded  2.8e-15  31.7855  1.4e-14    17.6726    16.9849
    180  1.0 graded  3.3e-15  50.4931  0.0e+00    30.9010    30.4407
    180  1.1 graded  3.1e-15  53.3407  7.1e-15    32.8705    31.4287
  largest PA-PVE rig over rigid twists at w = 1: 5.9e-14 mm

Checks. The data are as constructed: bone lengths match the template to 6×10−136 \times 10^{-13} mm, the pelvis lies at the midpoint of the hips, A keeps bone lengths and L scales them as planned; these checks held. The closed-form scales agree with numerical minimization to 5×10−95 \times 10^{-9}, within the 10−610^{-6} tolerance, the rotation agrees with SciPy on all 50 poses, and on those poses every alignment lowers the summed squared error (the smallest decrease is 88.4 mm²): the implementation checks held. All sixteen identities derived before any run held to 10−1210^{-12} mm or better.

What each alignment removes. A depth error of 200 mm is the whole absolute error and vanishes under every alignment. A scale error about the pelvis gives 44.4 mm of absolute and root-relative error; N-MPJPE and PA-MPJPE with scale remove it exactly, rigid PA-MPJPE leaves 41.6 mm (3D Human Pose Estimation’s example shows the same for a scale about the camera). A 15° rotation costs 92.3 mm, which a scale cannot remove (N-MPJPE 91.8 mm) and both Procrustes alignments remove exactly.

The root. Moving only the pelvis by 50 mm gives an absolute error of 50/17=2.9450/17 = 2.94 mm but a pelvis-rooted error of 800/17=47.06800/17 = 47.06 mm: root alignment copies the pelvis’s error onto the other 16 joints. With the mid-hip root, only the pelvis itself is wrong (2.94 mm over 17 joints), and on the 14-joint set, which leaves the pelvis out, the error is 0. The Procrustes alignments give 5.61 and 5.64 mm, between the absolute and the pelvis-rooted value; the hypothesis that the similarity-aligned error lies between them held.

One wrong joint. With only the wrist wrong by 100 mm, the absolute and root-relative errors are 100/17=5.88100/17 = 5.88 mm, and 100/14=7.14100/14 = 7.14 mm on 14 joints. PA-MPJPE is 12.15 mm rigid and 12.60 mm with scale, more than twice as large: the hypothesis that Procrustes alignment raises MPJPE above the root-relative error here held for both variants, which confirms by computation that PA-MPJPE is not bounded by the root-relative error. N-MPJPE (8.45 mm) is also above it. The limb-length bias shows the same: both PA values (12.57 and 11.49 mm) and N-MPJPE (12.27 mm) exceed the root-relative 10.84 mm.

N-MPJPE against root-relative error. The hypothesis that N-MPJPE is below the root-relative error held for the rotation (91.80 against 92.26 mm) and for ALL (113.94 against 116.70 mm), and failed for the articulation noise (37.87 against 37.63 mm) and the limb-length bias (12.27 against 10.84 mm). The least-squares scale lowers the sum of squared errors, as least squares guarantees, but not necessarily the mean of the distances. In our reading, for the limb-length bias this follows from the construction: the eight torso joints other than the pelvis (hips, spine, thorax, neck, head, shoulders) have zero error under the pelvis root, and any scale other than 1 gives them an error, which the least-squares scale accepts to shrink the squared errors of the limbs.

Rigid against similarity. Whether PA with scale is at or below rigid PA was computed, not predicted. It is below for S, A, L and ALL, equal to within 10−1310^{-13} mm for T and Rot (where both are zero), and above for R0 (by 0.028 mm) and W (by 0.45 mm): the extra freedom of scale lowers the squared error, and in these single-joint cases raises the mean distance.

Joint set and the combined case. Under articulation noise the 14-joint error is above the 17-joint error (44.16 against 37.63 mm), so the joint-set hypothesis held; in our reading, because two of the three joints the 14-joint set drops, pelvis and spine, have no error by construction, their positions depending only on the unperturbed root rotation, so the 14-joint mean is the larger whenever the thorax’s error is below three times that mean; under the limb-length bias the ratio is exactly 17/14, since those three joints have no error. With all components, absolute error 225.18 mm, root-relative 116.70 mm and PA with scale 39.80 mm are ordered as hypothesized, and that hypothesis held: in this construction, most of the error is where the skeleton is and how it is oriented, not its shape.

What joints cannot see. The joint MPJPE of the tube stays below 10−1410^{-14} mm for every twist and widening, while the per-vertex error grows to 80 mm, the tube’s diameter, at a rigid 180° twist; the graded twist moves the vertices less (50.49 mm at 180°), and the widening alone costs (w−1)r=4(w - 1) r = 4 mm. The per-vertex errors match their closed forms to 5×10−145 \times 10^{-14} mm: hypotheses HB1 to HB3 held. Rigid Procrustes alignment of the vertices removes a rigid twist exactly (the largest residual is 5.9×10−145.9 \times 10^{-14} mm) but leaves the widening (4.0 mm) and the graded twist (5.4 to 32.9 mm for twists of 30° or more), and similarity alignment leaves 3.43 mm of the widening, because a single scale cannot widen a tube without lengthening it: these hypotheses held. An isolated limb that turns as a rigid body is thus invisible to PA-PVE as well, and a twist is seen only where it is distributed along the limb.

How the example was planned and run. The identities, the comparisons and the Part B hypotheses were written from derivation before any script ran. One derivation in that first draft was wrong and was corrected before any run: under R0 the mid-hip 17-joint error is 50/1750/17, not 0, because the pelvis joint itself is one of the 17 and the offset is its error. The identity tolerances (10−910^{-9} mm) were written with the derivations; a planning script then computed the residuals of the identities, of the implementation checks and of the rigid-twist Procrustes alignment on the example’s data, and the implementation checks’ tolerances (for instance 10−610^{-6} for the scales) were set with those residuals in view. A planning script had also computed the joint error and PVE of a rigid 30° twist of a tube of another radius, a case with a closed form. Those parts therefore illustrate exact relations rather than test predictions; the comparisons (the root, single-joint, N-MPJPE, joint-set and combined hypotheses) and the PA-PVE hypotheses for graded twists and widening were not computed before the pilot. After the pilot, two printing changes were put to the editor and made before the run shown here. The pilot compared the two Procrustes variants as floating-point numbers, which reported a strict comparison of two values that are both zero up to rounding; those lines now print the difference and treat a difference within 10−910^{-9} mm as equality. The table’s four decimals could not decide whether rigid alignment removes a rigid twist to 10−910^{-9} mm, so a line printing that residual was added. Every other printed line is identical to the pilot’s, and two runs give identical output.

  • PCK and AUC in 3D. The fraction of joints within a distance threshold (150 mm in Rhodin et al.’s NPCK and in Moon et al.’s 3DPCK), and the area under that curve over thresholds, report the distribution of joint errors rather than their mean; 3D Human Pose Estimation summarizes their use on MPI-INF-3DHP.
  • MPJLE. Ionescu et al.’s localization error counts joints beyond a perceptual tolerance; integrated over the tolerance it gives a clipped mean joint error (Human3.6M).
  • Absolute trajectory error. The same pattern for camera trajectories: an RMS position error after a rigid or similarity alignment whose choice changes the number (absolute trajectory error).
  • Object keypoint similarity and 2D PCK. For 2D poses, errors are scaled by object size and turned into similarities or hit rates; object keypoint similarity, Percentage of Correct Keypoints and human pose estimation describe them.
  • Mean squared error. Least-squares alignment minimizes a squared error, while MPJPE reports unsquared distances; the gap between the two criteria is the source of the cases above where an alignment raises MPJPE.

Related

  • Human3.6M

    The laboratory motion-capture dataset of 3.6 million 3D human poses with synchronized video, depth and body scans, its capture record, licence, split and evaluation server, the paper's own error measures, and how its headline numbers are aggregated.

  • Parametric Human Body Models

    Statistical models that turn a small vector of shape and pose parameters into a full 3D body mesh; SCAPE and SMPL, linear blend skinning on a kinematic tree, shape and pose blend shapes, the hand, face and whole-body models MANO, FLAME and SMPL-X, fitting and regressing them from images, and their licence terms.

  • Keypoint Detection

    Locating a fixed set of named points, such as an animal's paws, a hand's fingertips or a car's headlights, on every object of a category, or on categories never seen in training; keypoint definitions, transfer from human pose models, multi-view labelling, category-agnostic methods, benchmarks for animals, hands, faces and vehicles, and the PCK and OKS metrics.

  • Mean Squared Error

    The average squared difference between a prediction and its reference, over pixels, channels or values, with its square root RMSE and the mean absolute error MAE; how they are computed for images, what their units and statistical meaning are, and why very different distortions can share one MSE.

  • Absolute Trajectory Error

    The root-mean-square distance between estimated and ground-truth camera positions after the two trajectories are aligned; how the alignment (SE(3), Sim(3), yaw-only, all poses or the first), timestamp association and reference frames change the number, and how it relates to the relative pose error and the KITTI odometry metric.

  • Human Pose Estimation

    Locating the body joints of every person in an image, such as shoulders, elbows, knees and ankles, to recover each person's 2D skeleton; keypoint definitions, part models, heatmap networks, top-down and bottom-up multi-person methods, and the PCKh and COCO keypoint metrics.

References

  1. Ionescu, C., Papava, D., Olaru, V. & Sminchisescu, C. (2014). Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(7), 1325–1339.
  2. Martinez, J., Hossain, R., Romero, J. & Little, J. J. (2017). A Simple yet Effective Baseline for 3d Human Pose Estimation. IEEE International Conference on Computer Vision (ICCV).
  3. Rhodin, H., Spörri, J., Katircioglu, I., Constantin, V., Meyer, F., Müller, E., Salzmann, M. & Fua, P. (2018). Learning Monocular 3D Human Pose Estimation from Multi-view Images. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  4. Pavlakos, G., Zhu, L., Zhou, X. & Daniilidis, K. (2018). Learning to Estimate 3D Human Pose and Shape from a Single Color Image. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  5. Pavllo, D., Feichtenhofer, C., Grangier, D. & Auli, M. (2019). 3D Human Pose Estimation in Video with Temporal Convolutions and Semi-Supervised Training. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  6. Moon, G., Chang, J. Y. & Lee, K. M. (2019). Camera Distance-aware Top-down Approach for 3D Multi-person Pose Estimation from a Single RGB Image. IEEE/CVF International Conference on Computer Vision (ICCV).