Evaluation & Metrics
Mean Per Joint Position Error
The average Euclidean distance between predicted and true 3D joints, and the family built on it; absolute and root-relative error, the choice of root and joint set, scale-normalized N-MPJPE, Procrustes-aligned PA-MPJPE, root position error and per-vertex error, with what each alignment removes and what it hides.
intermediate
The mean per joint position error (MPJPE) is the average straight-line distance, usually in millimetres, between the joints a 3D pose estimator predicts and the joints of the ground-truth skeleton. It is the headline number of 3D human pose estimation and of 3D hand pose. The formula is one line, but the number a paper reports depends on what is done to the prediction before the distances are taken: nothing (absolute error), a translation that puts the two roots together (root-relative error), a single scale factor (N-MPJPE), or a full Procrustes alignment (PA-MPJPE). It also depends on which joint is the root, which joints are averaged, and how frames are pooled. Each alignment removes a kind of error. It can also spread one joint’s error over the whole skeleton, or hide errors that a skeleton cannot represent. This article covers the family as a whole. The benchmark protocols built on it are summarized in 3D Human Pose Estimation and Human3.6M, and the alignment algorithm in the Kabsch algorithm.
Formula
For one person in one frame, with predicted joints and true joints in , and a transformation applied to the prediction,
Ionescu et al. (2014, Eq. 8) define MPJPE for a frame, with the root at the origin of the prediction’s coordinate system, and the error of a set of frames as the mean over frames; Human3.6M gives the definition and how published numbers are aggregated. The members of the family differ only in :
| Name (as printed) | Removes | Source | |
|---|---|---|---|
| absolute MPJPE (descriptive name); 3DPCK, “3DPCK without root alignment”, and MRPE for the root alone (Moon et al.) | identity | nothing | Moon et al. (2019) |
| MPJPE, protocol #1 | translation putting the predicted root on the true root | the person’s position | Martinez et al. (2017) |
| NMPJPE (Rhodin et al.), N-MPJPE (Pavllo et al.) | root alignment, then one scale | position and overall size | Rhodin et al. (2018); Pavllo et al. (2019) |
| PMPJPE, P-MPJPE, PA-MPJPE, reconstruction error | least-squares rotation and translation, with or without scale | position, orientation and size | see 3D Human Pose Estimation |
Root-relative error. With root joint , , so the error compares with . Martinez et al. (2017) describe the standard Human3.6M evaluation as the average error over all joints and cameras after aligning the root, “central hip”, joint, and call it protocol #1.
N-MPJPE. Rhodin et al. (2018) make the error “independent from the subjects’ height” by applying a single scale factor to the prediction so as to minimize the squared distance to the label; their poses are relative to the pelvis, and orientation is left unchanged. They call the result NMPJPE, and NPCK for the percentage of correct keypoints within 15 cm. With root-relative prediction and truth , the least-squares scale has a closed form (our derivation; the paper states the criterion in words):
Pavllo et al. (2019) list it as protocol 3, which “aligns predicted poses with the ground-truth only in scale (N-MPJPE)”, and use it for their semi-supervised experiments. The work they say they follow is their reference 45, a different 2018 paper by Rhodin, Salzmann and Fua (ECCV, “Unsupervised geometry-aware representation for 3D human pose estimation”, not read), not the CVPR paper of Rhodin et al. described above.
PA-MPJPE. The prediction is aligned by the rotation and translation, and in some implementations the scale, that minimize the summed squared joint distances, the problem the Kabsch algorithm and Umeyama’s extension solve; 3D Human Pose Estimation describes the reference implementation and the papers that leave the scale out.
Root position error. Moon et al. (2019, Eq. 5) introduce the mean root position error, MRPE , the mean over test samples of the distance between the estimated and true root coordinates, to evaluate absolute localization separately from root-relative pose.
Per-vertex error. Mesh methods replace joints by surface vertices of a common mesh. Pavlakos et al. (2018) report the “mean per-vertex error” between predicted and ground-truth shape on UP-3D and SURREAL. The paper does not print its formula; read as its name says, it is the same mean of distances over the vertices of the SMPL mesh.
All members have the units of the coordinates: millimetres in 3D, pixels for 2D joints.
Intuition
MPJPE asks how far, on average, each joint is from where it should be. Each alignment first moves the prediction as close to the truth as a chosen class of transformations allows, and then asks the same question. The larger the class, the more kinds of error are forgiven:
- No alignment charges everything, including where the person stands and how far from the camera, a distance that a single image does not record.
- Root alignment forgives the person’s position, but it treats the root joint as exact. Any error at the root is copied onto every other joint.
- Scale alignment also forgives the overall size of the person. A tall person far away and a short person near the camera can project to nearly the same image.
- Procrustes alignment also forgives orientation. What is left is the shape of the skeleton.
What none of them sees is anything a joint position cannot represent: the twist of a limb about its own axis, the width of the body, the surface. The per-vertex error measures those.
Interpretation
- Range. ; 0 means the aligned skeletons coincide. There is no upper bound and no normalization by body size, except in N-MPJPE and PA-MPJPE with scale.
- Units. Millimetres for 3D results on body benchmarks. A 50 mm MPJPE means the joints are, on average, 5 cm from the truth after the stated alignment. It does not mean every joint is within 5 cm. As a mean of distances, not squared distances, it is less dominated by one bad joint than a squared error such as mean squared error or its root, but it still grows linearly with that joint’s error (Human3.6M discusses the perceptual MPJLE that Ionescu et al. proposed for this reason).
- Comparability. Two MPJPE values are comparable only with the same alignment, root, joint set, frame sampling, camera set and aggregation. Martinez et al.’s Table 3 lists methods evaluated with 14 and with 17 joints side by side, marked “14j” and “17j”. Moon et al. (2019) number their Human3.6M protocols the other way round from Martinez et al.: their “Protocol 1” trains on six subjects, tests on S11 and reports PA MPJPE, and their “Protocol 2” tests on S9 and S11 with MPJPE after root alignment. 3D Human Pose Estimation collects further differences in what “Protocol #2” means.
- Ordering. In Rhodin et al.’s Table 1, every method with all three values has MPJPE above NMPJPE above PMPJPE. The ordering is not guaranteed. Each alignment minimizes a sum of squared distances, while MPJPE averages unsquared distances, so an alignment can raise MPJPE (see Failure Cases and the example).
When to Use It
- Root-relative MPJPE for monocular single-person methods that do not estimate the person’s distance from the camera, the setting in which Human3.6M results are usually reported.
- Absolute MPJPE, MRPE or 3DPCK when the position of the person matters: multi-person scenes, multi-view systems, and any downstream use in a real coordinate frame. Moon et al. (2019) introduce MRPE to evaluate the localization of the absolute root and report it beside root-relative MPJPE (their Table 1). According to 3D Human Pose Estimation, Iskakov et al. (2019) excluded scenes of one Human3.6M subject whose ground truth is shifted in 3D, an error that subtracting the pelvis hides, before reporting absolute error.
- N-MPJPE when the method cannot know the subject’s metric size. Rhodin et al. trained some networks with a scale-normalized pose distance (their NSE, Eq. 4, which divides each pose by its norm), and their Table 1 marks MPJPE “not applicable” for those networks while giving NMPJPE and PMPJPE.
- PA-MPJPE to compare articulated shape independently of global orientation, for instance across datasets whose camera conventions differ, keeping in mind that two variants (with and without scale) are in use.
- Per-vertex error when the output is a body mesh and the surface matters: clothing, shape, contact, avatars.
Papers report several at once: Rhodin et al.’s Table 1 has MPJPE, NMPJPE and PMPJPE columns; their supervised baseline trained with the plain squared error scores 66.8, 63.3 and 51.6 mm on Human3.6M (S9 and S11, 10 fps), in that order.
Failure Cases
- A root error becomes every joint’s error. Root alignment subtracts the predicted root from every predicted joint. If only the root is wrong, by , the absolute error is while the root-relative error is for joints, the root included with zero error: 16 times larger for 17 joints. A root defined from other joints, such as the midpoint of the hips, is unaffected by an error at the pelvis joint, though an error in one hip still moves it by half that error.
- Alignment spreads one joint’s error. If one wrist is wrong and everything else is right, root alignment leaves the error on the wrist. A least-squares alignment moves the whole skeleton toward the wrong wrist, giving every joint a small error and the wrist a slightly smaller one. Because MPJPE averages unsquared distances, the total can rise. The example’s single-wrist case has PA-MPJPE about twice the root-relative error.
- Scale normalization is not free either. The least-squares scale lowers the squared error but can raise MPJPE when the error is concentrated in some joints, as in the example’s limb-length bias.
- Twist and surface are invisible. Joint positions do not record a limb’s rotation about its own axis, and a skeleton has no width. A forearm turned by 180° about its axis, or a body 10% wider, has zero MPJPE as long as its joints stay where they were, as in the example’s tube; the per-vertex error sees both. Kanazawa et al. (2018) observe, as 3D Human Pose Estimation reports, that MPJPE does not correlate well with the visual quality of results.
- Alignment of the evaluation, not of the method. PA-MPJPE uses the ground truth to align each prediction, information no deployed system has. A method can score well in PA-MPJPE with a wrong orientation and scale.
- Hidden conventions. Whether the root is counted in the mean (Human3.6M shows that 3d-pose-baseline’s 17-joint mean with a zero root error is exactly 16/17 of the 16-joint mean), which joints are kept, and whether frames or actions are averaged all move the number; in Human3.6M’s example the root convention alone moves it by 2.6 mm.
Variants
- Joint sets and roots. Pavllo et al. (2019) adopt a 17-joint skeleton on Human3.6M following previous work, and 3D Human Pose Estimation notes that the root is usually the pelvis; Martinez et al.’s Table 3 also lists 14-joint results. Rhodin et al. predict 16 joints relative to the pelvis. The root is the pelvis or central hip in these papers.
- 2D MPJPE. The same mean over image coordinates, in pixels; Ionescu et al. state that in 2D the error is reported in pixels, and Pavllo et al. use a “2D MPJPE loss” on reprojected joints for their semi-supervised training.
- Weighted MPJPE (WMPJPE). Pavllo et al. (2019, Eq. 1) weight the trajectory error of each sample by the inverse ground-truth depth, as a training loss for the global trajectory: regressing a precise trajectory for distant subjects is harder and, for back-projection, unnecessary.
- MPJPE for hands. Keypoint Detection describes InterHand2.6M’s MPJPE after aligning each hand’s root, and its mean relative-root position error (MRRPE), the error of the left hand’s root relative to the right hand’s.
- Per-vertex error (PVE) and its aligned form. Pavlakos et al. (2018) evaluate mesh recovery by mean per-vertex error; in their UP-3D ablation (Table 1), adding a per-vertex loss to the rotation-matrix parameter loss lowers it from 140.7 to 120.7 mm with Priors trained on UP-3D. They give the reason that the loss correlates better with the vertex-to-vertex error used for evaluation. Parametric Human Body Models reports SMPLify-X’s vertex-to-vertex error after Procrustes alignment, the per-vertex analogue of PA-MPJPE.
- Velocity and acceleration errors, UMPJPE, MPJAE, MPJLE. Different measures with similar names: velocity and acceleration errors are described in 3D Human Pose Estimation, and the universal, angle and localization variants of Ionescu et al. in Human3.6M.
Example
The example builds predictions from known error components and prints what each version of the metric makes of them. Part A poses 200 random skeletons of 17 joints in Human3.6M’s joint order (bone lengths of 115 to 445 mm, random local rotations of the heading, spine, neck, hips, knees, shoulders and elbows). The pelvis is placed at , , mm in a right-handed frame with up; the frame matters only for the absolute error. Each prediction is the ground truth with one component of error, then with all of them:
- T: the whole pose moved by mm, a depth error;
- R0: the pelvis joint alone moved by 50 mm in a random direction, a root-localization error;
- S: the pose scaled by 0.9 about the pelvis;
- Rot: a rotation by 15° about a random axis through the pelvis;
- A: articulation noise, a random rotation vector with 4° standard deviation per component multiplied onto the local rotation of every non-root joint that has a child (bone lengths unchanged);
- L: a limb-length bias, thighs and shins 5% too long, upper arms and forearms 5% too short;
- W: the right wrist alone moved by 100 mm in a random direction;
- ALL: A and L in the kinematics, then S, Rot, T, R0 and W in that order.
Seven measures are printed as means over the 200 poses: absolute error (abs17); root-relative error with the pelvis as root (rr17p) and with the midpoint of the hips as root (rr17h); the mid-hip error on 14 joints without pelvis, spine and thorax (rr14h); N-MPJPE with Rhodin et al.’s least-squares scale on pelvis-relative poses (N17p); and PA-MPJPE with a rigid alignment (PArig) and with a similarity alignment (PAsim), both centred on the centroids with reflections excluded. The scale and alignment are checked against SciPy’s minimize_scalar and orthogonal_procrustes, and the script checks that each alignment’s sum of squared errors is not above the root-aligned one, which least squares guarantees.
Part B is a forearm-like tube of radius 40 mm and length 250 mm along , with 26 rings of 24 vertices; its two joints, elbow and wrist, are the centroids of the end rings, a stand-in for the joint regressor of a body model. It is twisted about its axis by 0 to 180°, either rigidly or by an angle growing linearly from the elbow to the wrist, and widened by ; the joint MPJPE, the per-vertex error (PVE), its difference from the closed form, and the PVE after rigid and similarity Procrustes alignment of the 624 vertices are printed. The script uses NumPy 2.3.5 and SciPy 1.16.3 (Python 3.13) and runs in about half a second.
import os
for v in ("OMP_NUM_THREADS", "OPENBLAS_NUM_THREADS", "MKL_NUM_THREADS", "VECLIB_MAXIMUM_THREADS"):
os.environ[v] = "1" # one BLAS thread, set before NumPy is imported
import numpy as np
from scipy.linalg import orthogonal_procrustes
from scipy.optimize import minimize_scalar
from scipy.spatial.transform import Rotation
# 17 joints in Human3.6M order; parent, rest direction (y up, person faces +z), length in mm
NAMES = ["pelvis", "r_hip", "r_knee", "r_ankle", "l_hip", "l_knee", "l_ankle", "spine", "thorax",
"neck", "head", "l_shoulder", "l_elbow", "l_wrist", "r_shoulder", "r_elbow", "r_wrist"]
BONES = {1: (0, (-1, 0, 0), 135), 2: (1, (0, -1, 0), 445), 3: (2, (0, -1, 0), 435),
4: (0, (1, 0, 0), 135), 5: (4, (0, -1, 0), 445), 6: (5, (0, -1, 0), 435),
7: (0, (0, 1, 0), 235), 8: (7, (0, 1, 0), 245), 9: (8, (0, 1, 0), 115),
10: (9, (0, 1, 0), 115), 11: (8, (1, 0, 0), 165), 12: (11, (0, -1, 0), 285),
13: (12, (0, -1, 0), 255), 14: (8, (-1, 0, 0), 165), 15: (14, (0, -1, 0), 285),
16: (15, (0, -1, 0), 255)}
LIMB = {2: 1.05, 3: 1.05, 5: 1.05, 6: 1.05, 12: 0.95, 13: 0.95, 15: 0.95, 16: 0.95} # L: bone -> factor
HAS_CHILD = sorted({p for p, _, _ in BONES.values()} - {0})
J14 = [j for j in range(17) if j not in (0, 7, 8)] # drop pelvis, spine, thorax
N_POSES = 200
def R_(axis, deg):
return Rotation.from_euler(axis, deg, degrees=True).as_matrix()
def sample_angles(rng):
"""Local rotation of each joint (3x3) for one random pose."""
L = {j: np.eye(3) for j in range(17)}
L[0] = R_("yxz", [rng.uniform(-180, 180), rng.uniform(-10, 10), rng.uniform(-10, 10)])
L[7] = R_("yxz", [rng.uniform(-30, 30), rng.uniform(-10, 30), rng.uniform(-15, 15)])
L[9] = R_("yx", [rng.uniform(-45, 45), rng.uniform(-20, 30)])
for hip, knee in ((1, 2), (4, 5)):
L[hip] = R_("yxz", [rng.uniform(-20, 20), rng.uniform(-100, 30), rng.uniform(-20, 20)])
L[knee] = R_("x", rng.uniform(0, 130))
for sh, el, side in ((11, 12, 1), (14, 15, -1)):
L[sh] = R_("yxz", [rng.uniform(-30, 30), rng.uniform(-70, 40), side * rng.uniform(0, 70)])
L[el] = R_("x", -rng.uniform(0, 140))
return L
def fk(L, root, limb=None):
"""Forward kinematics: joint positions (17, 3) in mm, pelvis at `root`."""
X = np.zeros((17, 3)); X[0] = root; Rw = {0: L[0]}
for j in range(1, 17):
p, d, length = BONES[j]
f = limb.get(j, 1.0) if limb else 1.0
X[j] = X[p] + Rw[p] @ (f * length * np.array(d, float))
Rw[j] = Rw[p] @ L[j]
return X
def unit(rng):
v = rng.normal(size=3); return v / np.linalg.norm(v)
# ---------- measures ----------
def mean_norm(P, G): return np.linalg.norm(P - G, axis=-1).mean()
def rel(X, root): return X - (X[0] if root == "pelvis" else 0.5 * (X[1] + X[4]))
def n_scale(p, g):
"""Rhodin et al.: one scale on the prediction minimizing the squared distance."""
return (p * g).sum() / (p * p).sum()
def procrustes(p, g, scale):
"""Similarity (scale=True) or rigid alignment of p onto g; returns aligned p."""
mp, mg = p.mean(0), g.mean(0)
p0, g0 = p - mp, g - mg
U, s, Vt = np.linalg.svd(p0.T @ g0)
D = np.diag([1, 1, np.sign(np.linalg.det(U @ Vt))]) # no reflections
Rm = U @ D @ Vt # p0 @ Rm ~ g0
a = (s * np.diag(D)).sum() / (p0 ** 2).sum() if scale else 1.0
return a * p0 @ Rm + mg, Rm, a
def measures(P, G):
pp, gp = rel(P, "pelvis"), rel(G, "pelvis")
ph, gh = rel(P, "midhip"), rel(G, "midhip")
return np.array([
mean_norm(P, G), # abs17
mean_norm(pp, gp), # rr17p
mean_norm(ph, gh), # rr17h
mean_norm(ph[J14], gh[J14]), # rr14h
mean_norm(n_scale(pp, gp) * pp, gp), # N17p
mean_norm(procrustes(P, G, False)[0], G), # PArig17
mean_norm(procrustes(P, G, True)[0], G)]) # PAsim17
COLS = ["abs17", "rr17p", "rr17h", "rr14h", "N17p", "PArig", "PAsim"]
# ---------- data ----------
rng = np.random.default_rng(0)
poses = []
for i in range(N_POSES):
L = sample_angles(rng)
root = np.array([rng.uniform(-500, 500), 900.0, rng.uniform(3500, 5500)])
poses.append(dict(L=L, root=root, G=fk(L, root),
artic={j: Rotation.from_rotvec(np.radians(4) * rng.normal(size=3)).as_matrix()
for j in HAS_CHILD},
d_root=50 * unit(rng), d_wrist=100 * unit(rng),
rot=Rotation.from_rotvec(np.radians(15) * unit(rng)).as_matrix()))
T = np.array([0.0, 0.0, 200.0])
def predict(ps, comps):
L, root, G = ps["L"], ps["root"], ps["G"]
if "A" in comps or "L" in comps:
La = {j: (L[j] @ ps["artic"][j] if ("A" in comps and j in ps["artic"]) else L[j]) for j in L}
P = fk(La, root, LIMB if "L" in comps else None)
else:
P = G.copy()
if "S" in comps: P = root + 0.9 * (P - root)
if "Rot" in comps: P = root + (P - root) @ ps["rot"].T
if "T" in comps: P = P + T
if "R0" in comps: P[0] += ps["d_root"]
if "W" in comps: P[16] += ps["d_wrist"]
return P
CONDS = [("T", {"T"}), ("R0", {"R0"}), ("S", {"S"}), ("Rot", {"Rot"}), ("A", {"A"}),
("L", {"L"}), ("W", {"W"}), ("ALL", {"A", "L", "S", "Rot", "T", "R0", "W"})]
# ---------- checks on the data and the implementations ----------
blen = np.array([[np.linalg.norm(p["G"][j] - p["G"][BONES[j][0]]) - BONES[j][2] for j in BONES] for p in poses])
mid = max(np.linalg.norm(rel(p["G"], "midhip")[0]) for p in poses)
PA_ = [predict(p, {"A"}) for p in poses]
blenA = max(abs(np.linalg.norm(P[j] - P[BONES[j][0]]) - BONES[j][2]) for P in PA_ for j in BONES)
PL_ = [predict(p, {"L"}) for p in poses]
blenL = max(abs(np.linalg.norm(P[j] - P[BONES[j][0]]) - LIMB.get(j, 1.0) * BONES[j][2]) for P in PL_ for j in BONES)
print("data checks")
print(f" {N_POSES} poses x 17 joints; ground-truth bone length error {np.abs(blen).max():.1e} mm, "
f"pelvis - mid-hip {mid:.1e} mm")
print(f" A keeps bone lengths: {blenA:.1e} mm; L scales them as planned: {blenL:.1e} mm")
g_n = g_s = 0.0; d_rot = 0.0; n_rot = 0; sse_viol = np.inf
for p in poses[:50]:
P, G = predict(p, {"A", "L", "S", "Rot", "R0"}), p["G"]
pp, gp = rel(P, "pelvis"), rel(G, "pelvis")
s_opt = minimize_scalar(lambda s: ((s * pp - gp) ** 2).sum(), bracket=(0.5, 1.5), tol=1e-12).x
g_n = max(g_n, abs(n_scale(pp, gp) - s_opt))
Al, Rm, a = procrustes(P, G, True)
a_opt = minimize_scalar(lambda s: ((s * (P - P.mean(0)) @ Rm - (G - G.mean(0))) ** 2).sum(),
bracket=(0.5, 1.5), tol=1e-12).x
g_s = max(g_s, abs(a - a_opt))
Rs = orthogonal_procrustes(P - P.mean(0), G - G.mean(0))[0]
if np.linalg.det(Rs) > 0: # scipy allows reflections; compare proper rotations
d_rot = max(d_rot, np.abs(Rs - Rm).max()); n_rot += 1
sse = lambda A, B: ((A - B) ** 2).sum()
sse_viol = min(sse_viol, sse(pp, gp) - sse(Al, G), sse(pp, gp) - sse(procrustes(P, G, False)[0], G),
sse(pp, gp) - sse(n_scale(pp, gp) * pp, gp))
print("implementation checks (50 poses, condition A+L+S+Rot+R0)")
print(f" N scale vs minimize_scalar {g_n:.1e}; PA scale vs minimize_scalar {g_s:.1e}; "
f"PA rotation vs scipy orthogonal_procrustes {d_rot:.1e} ({n_rot} of 50 poses compared)")
print(f" smallest SSE(root-aligned) - SSE(aligned) over N, PA rigid, PA sim: {sse_viol:.3e} mm^2 (>= 0 expected)")
# ---------- part A: what each alignment removes ----------
res = {}
for name, comps in CONDS:
res[name] = np.mean([measures(predict(p, comps), p["G"]) for p in poses], axis=0)
print("\nA. mean over 200 poses (mm)")
print(f" {'cond':4s} " + " ".join(f"{c:>9s}" for c in COLS))
for name, _ in CONDS:
print(f" {name:4s} " + " ".join(f"{v:9.4f}" for v in res[name]))
g0 = np.mean([np.linalg.norm(rel(p["G"], "pelvis"), axis=1).mean() for p in poses])
print("\nA. identities (difference from the derived value, mm)")
ident = [("T abs - 200", res["T"][0] - 200), ("T max(rr17p..PAsim)", res["T"][1:].max()),
("R0 abs - 50/17", res["R0"][0] - 50 / 17), ("R0 rr17p - 800/17", res["R0"][1] - 800 / 17),
("R0 rr17h - 50/17", res["R0"][2] - 50 / 17), ("R0 rr14h", res["R0"][3]),
("S abs - rr17p", res["S"][0] - res["S"][1]), ("S abs - 0.1*mean|g-g0|", res["S"][0] - 0.1 * g0),
("S max(N, PAsim)", max(res["S"][4], res["S"][6])),
("Rot abs - rr17p", res["Rot"][0] - res["Rot"][1]), ("Rot max(PArig, PAsim)", res["Rot"][5:].max()),
("W abs - 100/17", res["W"][0] - 100 / 17), ("W rr17p - 100/17", res["W"][1] - 100 / 17),
("W rr17h - 100/17", res["W"][2] - 100 / 17), ("W rr14h - 100/14", res["W"][3] - 100 / 14),
("L rr14h - 17/14*rr17h", res["L"][3] - 17 / 14 * res["L"][2])]
for k, v in ident:
print(f" {k:24s} {v: .1e} {'held' if abs(v) <= 1e-9 else 'FAILED'}")
print("\nA. comparisons (mm)")
r = res
cmp_ = [("H2 R0: abs < PAsim < rr17p", f"{r['R0'][0]:.4f} < {r['R0'][6]:.4f} < {r['R0'][1]:.4f}",
r["R0"][0] < r["R0"][6] < r["R0"][1]),
("H3 W: PArig > rr17p", f"{r['W'][5]:.4f} > {r['W'][1]:.4f}", r["W"][5] > r["W"][1]),
("H3 W: PAsim > rr17p", f"{r['W'][6]:.4f} > {r['W'][1]:.4f}", r["W"][6] > r["W"][1])]
for c in ("Rot", "A", "L", "ALL"):
cmp_.append((f"H4 {c}: N17p < rr17p", f"{r[c][4]:.4f} < {r[c][1]:.4f}", r[c][4] < r[c][1]))
for c, _ in CONDS: # equal within 1e-9 mm counts as <=
d = r[c][6] - r[c][5]
cmp_.append((f"H5 {c}: PAsim <= PArig", f"PAsim - PArig = {d: .1e}", d <= 1e-9))
cmp_ += [("H6 A: rr14h > rr17h", f"{r['A'][3]:.4f} > {r['A'][2]:.4f}", r["A"][3] > r["A"][2]),
("H7 ALL: abs > rr17p > PAsim", f"{r['ALL'][0]:.4f} > {r['ALL'][1]:.4f} > {r['ALL'][6]:.4f}",
r["ALL"][0] > r["ALL"][1] > r["ALL"][6])]
for k, s, ok in cmp_:
print(f" {k:28s} {s:32s} {'true' if ok else 'false'}")
# ---------- part B: what joints cannot see ----------
RAD, LEN, NR, NV = 40.0, 250.0, 26, 24 # forearm tube, mm; rings x vertices per ring
xk = np.linspace(0, LEN, NR)
th = 2 * np.pi * np.arange(NV) / NV
V = np.stack([np.repeat(xk, NV), RAD * np.tile(np.cos(th), NR), RAD * np.tile(np.sin(th), NR)], 1)
def joints(V): # elbow, wrist = centroids of the end rings
return np.stack([V[:NV].mean(0), V[-NV:].mean(0)])
Jt = joints(V)
def deform(V, phi, w, graded):
ang = np.radians(phi) * (V[:, 0] / LEN if graded else 1.0)
c, s = np.cos(ang), np.sin(ang)
y, z = w * V[:, 1], w * V[:, 2]
return np.stack([V[:, 0], c * y - s * z, s * y + c * z], 1)
print("\nB. forearm tube (r = 40 mm, 250 mm, 26 x 24 vertices): joint MPJPE and per-vertex error (mm)")
print(f" {'twist':>5s} {'w':>4s} {'mode':>6s} {'MPJPE':>8s} {'PVE':>8s} {'|PVE-an|':>8s} {'PA-PVE rig':>10s} {'PA-PVE sim':>10s}")
pa_rig_max = 0.0
for graded in (False, True):
for phi in (0, 30, 90, 180):
for w in (1.0, 1.1):
Vd = deform(V, phi, w, graded)
Jd = joints(Vd)
ang = np.radians(phi) * (xk / LEN if graded else np.ones(NR))
an = np.mean(RAD * np.sqrt(w * w + 1 - 2 * w * np.cos(ang)))
pve = mean_norm(Vd, V)
print(f" {phi:5d} {w:4.1f} {'graded' if graded else 'rigid':>6s} {mean_norm(Jd, Jt):8.1e} {pve:8.4f} "
f"{abs(pve - an):8.1e} {mean_norm(procrustes(Vd, V, False)[0], V):10.4f} "
f"{mean_norm(procrustes(Vd, V, True)[0], V):10.4f}")
if not graded and w == 1.0:
pa_rig_max = max(pa_rig_max, mean_norm(procrustes(Vd, V, False)[0], V))
print(f" largest PA-PVE rig over rigid twists at w = 1: {pa_rig_max:.1e} mm")
data checks
200 poses x 17 joints; ground-truth bone length error 5.7e-13 mm, pelvis - mid-hip 4.5e-13 mm
A keeps bone lengths: 6.0e-13 mm; L scales them as planned: 5.7e-13 mm
implementation checks (50 poses, condition A+L+S+Rot+R0)
N scale vs minimize_scalar 4.8e-09; PA scale vs minimize_scalar 1.3e-09; PA rotation vs scipy orthogonal_procrustes 0.0e+00 (50 of 50 poses compared)
smallest SSE(root-aligned) - SSE(aligned) over N, PA rigid, PA sim: 8.841e+01 mm^2 (>= 0 expected)
A. mean over 200 poses (mm)
cond abs17 rr17p rr17h rr14h N17p PArig PAsim
T 200.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
R0 2.9412 47.0588 2.9412 0.0000 45.2496 5.6073 5.6353
S 44.4459 44.4459 44.4459 48.9023 0.0000 41.5644 0.0000
Rot 92.2599 92.2599 92.2599 101.4012 91.8045 0.0000 0.0000
A 37.6273 37.6273 37.6273 44.1559 37.8708 31.0476 30.9734
L 10.8380 10.8380 10.8380 13.1604 12.2727 12.5695 11.4928
W 5.8824 5.8824 5.8824 7.1429 8.4512 12.1548 12.6039
ALL 225.1800 116.7046 109.6868 118.0628 113.9411 54.0453 39.8025
A. identities (difference from the derived value, mm)
T abs - 200 0.0e+00 held
T max(rr17p..PAsim) 3.8e-13 held
R0 abs - 50/17 2.2e-15 held
R0 rr17p - 800/17 4.3e-14 held
R0 rr17h - 50/17 2.2e-15 held
R0 rr14h 0.0e+00 held
S abs - rr17p 0.0e+00 held
S abs - 0.1*mean|g-g0| 0.0e+00 held
S max(N, PAsim) 8.6e-13 held
Rot abs - rr17p 0.0e+00 held
Rot max(PArig, PAsim) 8.6e-13 held
W abs - 100/17 8.9e-16 held
W rr17p - 100/17 8.9e-16 held
W rr17h - 100/17 8.9e-16 held
W rr14h - 100/14 -1.8e-14 held
L rr14h - 17/14*rr17h 7.1e-15 held
A. comparisons (mm)
H2 R0: abs < PAsim < rr17p 2.9412 < 5.6353 < 47.0588 true
H3 W: PArig > rr17p 12.1548 > 5.8824 true
H3 W: PAsim > rr17p 12.6039 > 5.8824 true
H4 Rot: N17p < rr17p 91.8045 < 92.2599 true
H4 A: N17p < rr17p 37.8708 < 37.6273 false
H4 L: N17p < rr17p 12.2727 < 10.8380 false
H4 ALL: N17p < rr17p 113.9411 < 116.7046 true
H5 T: PAsim <= PArig PAsim - PArig = 2.4e-14 true
H5 R0: PAsim <= PArig PAsim - PArig = 2.8e-02 false
H5 S: PAsim <= PArig PAsim - PArig = -4.2e+01 true
H5 Rot: PAsim <= PArig PAsim - PArig = 1.6e-14 true
H5 A: PAsim <= PArig PAsim - PArig = -7.4e-02 true
H5 L: PAsim <= PArig PAsim - PArig = -1.1e+00 true
H5 W: PAsim <= PArig PAsim - PArig = 4.5e-01 false
H5 ALL: PAsim <= PArig PAsim - PArig = -1.4e+01 true
H6 A: rr14h > rr17h 44.1559 > 37.6273 true
H7 ALL: abs > rr17p > PAsim 225.1800 > 116.7046 > 39.8025 true
B. forearm tube (r = 40 mm, 250 mm, 26 x 24 vertices): joint MPJPE and per-vertex error (mm)
twist w mode MPJPE PVE |PVE-an| PA-PVE rig PA-PVE sim
0 1.0 rigid 0.0e+00 0.0000 0.0e+00 0.0000 0.0000
0 1.1 rigid 1.7e-15 4.0000 4.5e-14 4.0000 3.4285
30 1.0 rigid 3.2e-15 20.7055 3.6e-15 0.0000 0.0000
30 1.1 rigid 3.3e-15 22.0815 1.1e-14 4.0000 3.4285
90 1.0 rigid 6.5e-15 56.5685 7.1e-15 0.0000 0.0000
90 1.1 rigid 4.0e-15 59.4643 1.4e-14 4.0000 3.4285
180 1.0 rigid 6.6e-15 80.0000 0.0e+00 0.0000 0.0000
180 1.1 rigid 4.6e-15 84.0000 0.0e+00 4.0000 3.4285
0 1.0 graded 0.0e+00 0.0000 0.0e+00 0.0000 0.0000
0 1.1 graded 1.7e-15 4.0000 4.5e-14 4.0000 3.4285
30 1.0 graded 1.6e-15 10.4099 3.6e-15 5.4370 5.4350
30 1.1 graded 2.5e-15 12.0223 2.7e-14 7.2816 6.7792
90 1.0 graded 3.2e-15 29.7719 0.0e+00 16.1111 16.0533
90 1.1 graded 2.8e-15 31.7855 1.4e-14 17.6726 16.9849
180 1.0 graded 3.3e-15 50.4931 0.0e+00 30.9010 30.4407
180 1.1 graded 3.1e-15 53.3407 7.1e-15 32.8705 31.4287
largest PA-PVE rig over rigid twists at w = 1: 5.9e-14 mm
Checks. The data are as constructed: bone lengths match the template to mm, the pelvis lies at the midpoint of the hips, A keeps bone lengths and L scales them as planned; these checks held. The closed-form scales agree with numerical minimization to , within the tolerance, the rotation agrees with SciPy on all 50 poses, and on those poses every alignment lowers the summed squared error (the smallest decrease is 88.4 mm²): the implementation checks held. All sixteen identities derived before any run held to mm or better.
What each alignment removes. A depth error of 200 mm is the whole absolute error and vanishes under every alignment. A scale error about the pelvis gives 44.4 mm of absolute and root-relative error; N-MPJPE and PA-MPJPE with scale remove it exactly, rigid PA-MPJPE leaves 41.6 mm (3D Human Pose Estimation’s example shows the same for a scale about the camera). A 15° rotation costs 92.3 mm, which a scale cannot remove (N-MPJPE 91.8 mm) and both Procrustes alignments remove exactly.
The root. Moving only the pelvis by 50 mm gives an absolute error of mm but a pelvis-rooted error of mm: root alignment copies the pelvis’s error onto the other 16 joints. With the mid-hip root, only the pelvis itself is wrong (2.94 mm over 17 joints), and on the 14-joint set, which leaves the pelvis out, the error is 0. The Procrustes alignments give 5.61 and 5.64 mm, between the absolute and the pelvis-rooted value; the hypothesis that the similarity-aligned error lies between them held.
One wrong joint. With only the wrist wrong by 100 mm, the absolute and root-relative errors are mm, and mm on 14 joints. PA-MPJPE is 12.15 mm rigid and 12.60 mm with scale, more than twice as large: the hypothesis that Procrustes alignment raises MPJPE above the root-relative error here held for both variants, which confirms by computation that PA-MPJPE is not bounded by the root-relative error. N-MPJPE (8.45 mm) is also above it. The limb-length bias shows the same: both PA values (12.57 and 11.49 mm) and N-MPJPE (12.27 mm) exceed the root-relative 10.84 mm.
N-MPJPE against root-relative error. The hypothesis that N-MPJPE is below the root-relative error held for the rotation (91.80 against 92.26 mm) and for ALL (113.94 against 116.70 mm), and failed for the articulation noise (37.87 against 37.63 mm) and the limb-length bias (12.27 against 10.84 mm). The least-squares scale lowers the sum of squared errors, as least squares guarantees, but not necessarily the mean of the distances. In our reading, for the limb-length bias this follows from the construction: the eight torso joints other than the pelvis (hips, spine, thorax, neck, head, shoulders) have zero error under the pelvis root, and any scale other than 1 gives them an error, which the least-squares scale accepts to shrink the squared errors of the limbs.
Rigid against similarity. Whether PA with scale is at or below rigid PA was computed, not predicted. It is below for S, A, L and ALL, equal to within mm for T and Rot (where both are zero), and above for R0 (by 0.028 mm) and W (by 0.45 mm): the extra freedom of scale lowers the squared error, and in these single-joint cases raises the mean distance.
Joint set and the combined case. Under articulation noise the 14-joint error is above the 17-joint error (44.16 against 37.63 mm), so the joint-set hypothesis held; in our reading, because two of the three joints the 14-joint set drops, pelvis and spine, have no error by construction, their positions depending only on the unperturbed root rotation, so the 14-joint mean is the larger whenever the thorax’s error is below three times that mean; under the limb-length bias the ratio is exactly 17/14, since those three joints have no error. With all components, absolute error 225.18 mm, root-relative 116.70 mm and PA with scale 39.80 mm are ordered as hypothesized, and that hypothesis held: in this construction, most of the error is where the skeleton is and how it is oriented, not its shape.
What joints cannot see. The joint MPJPE of the tube stays below mm for every twist and widening, while the per-vertex error grows to 80 mm, the tube’s diameter, at a rigid 180° twist; the graded twist moves the vertices less (50.49 mm at 180°), and the widening alone costs mm. The per-vertex errors match their closed forms to mm: hypotheses HB1 to HB3 held. Rigid Procrustes alignment of the vertices removes a rigid twist exactly (the largest residual is mm) but leaves the widening (4.0 mm) and the graded twist (5.4 to 32.9 mm for twists of 30° or more), and similarity alignment leaves 3.43 mm of the widening, because a single scale cannot widen a tube without lengthening it: these hypotheses held. An isolated limb that turns as a rigid body is thus invisible to PA-PVE as well, and a twist is seen only where it is distributed along the limb.
How the example was planned and run. The identities, the comparisons and the Part B hypotheses were written from derivation before any script ran. One derivation in that first draft was wrong and was corrected before any run: under R0 the mid-hip 17-joint error is , not 0, because the pelvis joint itself is one of the 17 and the offset is its error. The identity tolerances ( mm) were written with the derivations; a planning script then computed the residuals of the identities, of the implementation checks and of the rigid-twist Procrustes alignment on the example’s data, and the implementation checks’ tolerances (for instance for the scales) were set with those residuals in view. A planning script had also computed the joint error and PVE of a rigid 30° twist of a tube of another radius, a case with a closed form. Those parts therefore illustrate exact relations rather than test predictions; the comparisons (the root, single-joint, N-MPJPE, joint-set and combined hypotheses) and the PA-PVE hypotheses for graded twists and widening were not computed before the pilot. After the pilot, two printing changes were put to the editor and made before the run shown here. The pilot compared the two Procrustes variants as floating-point numbers, which reported a strict comparison of two values that are both zero up to rounding; those lines now print the difference and treat a difference within mm as equality. The table’s four decimals could not decide whether rigid alignment removes a rigid twist to mm, so a line printing that residual was added. Every other printed line is identical to the pilot’s, and two runs give identical output.
Related Metrics
- PCK and AUC in 3D. The fraction of joints within a distance threshold (150 mm in Rhodin et al.’s NPCK and in Moon et al.’s 3DPCK), and the area under that curve over thresholds, report the distribution of joint errors rather than their mean; 3D Human Pose Estimation summarizes their use on MPI-INF-3DHP.
- MPJLE. Ionescu et al.’s localization error counts joints beyond a perceptual tolerance; integrated over the tolerance it gives a clipped mean joint error (Human3.6M).
- Absolute trajectory error. The same pattern for camera trajectories: an RMS position error after a rigid or similarity alignment whose choice changes the number (absolute trajectory error).
- Object keypoint similarity and 2D PCK. For 2D poses, errors are scaled by object size and turned into similarities or hit rates; object keypoint similarity, Percentage of Correct Keypoints and human pose estimation describe them.
- Mean squared error. Least-squares alignment minimizes a squared error, while MPJPE reports unsquared distances; the gap between the two criteria is the source of the cases above where an alignment raises MPJPE.
Related
- Human3.6M
The laboratory motion-capture dataset of 3.6 million 3D human poses with synchronized video, depth and body scans, its capture record, licence, split and evaluation server, the paper's own error measures, and how its headline numbers are aggregated.
- Parametric Human Body Models
Statistical models that turn a small vector of shape and pose parameters into a full 3D body mesh; SCAPE and SMPL, linear blend skinning on a kinematic tree, shape and pose blend shapes, the hand, face and whole-body models MANO, FLAME and SMPL-X, fitting and regressing them from images, and their licence terms.
- Keypoint Detection
Locating a fixed set of named points, such as an animal's paws, a hand's fingertips or a car's headlights, on every object of a category, or on categories never seen in training; keypoint definitions, transfer from human pose models, multi-view labelling, category-agnostic methods, benchmarks for animals, hands, faces and vehicles, and the PCK and OKS metrics.
- Mean Squared Error
The average squared difference between a prediction and its reference, over pixels, channels or values, with its square root RMSE and the mean absolute error MAE; how they are computed for images, what their units and statistical meaning are, and why very different distortions can share one MSE.
- Absolute Trajectory Error
The root-mean-square distance between estimated and ground-truth camera positions after the two trajectories are aligned; how the alignment (SE(3), Sim(3), yaw-only, all poses or the first), timestamp association and reference frames change the number, and how it relates to the relative pose error and the KITTI odometry metric.
- Human Pose Estimation
Locating the body joints of every person in an image, such as shoulders, elbows, knees and ankles, to recover each person's 2D skeleton; keypoint definitions, part models, heatmap networks, top-down and bottom-up multi-person methods, and the PCKh and COCO keypoint metrics.
References
- Ionescu, C., Papava, D., Olaru, V. & Sminchisescu, C. (2014). Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(7), 1325–1339.
- Martinez, J., Hossain, R., Romero, J. & Little, J. J. (2017). A Simple yet Effective Baseline for 3d Human Pose Estimation. IEEE International Conference on Computer Vision (ICCV).
- Rhodin, H., Spörri, J., Katircioglu, I., Constantin, V., Meyer, F., Müller, E., Salzmann, M. & Fua, P. (2018). Learning Monocular 3D Human Pose Estimation from Multi-view Images. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
- Pavlakos, G., Zhu, L., Zhou, X. & Daniilidis, K. (2018). Learning to Estimate 3D Human Pose and Shape from a Single Color Image. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
- Pavllo, D., Feichtenhofer, C., Grangier, D. & Auli, M. (2019). 3D Human Pose Estimation in Video with Temporal Convolutions and Semi-Supervised Training. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
- Moon, G., Chang, J. Y. & Lee, K. M. (2019). Camera Distance-aware Top-down Approach for 3D Multi-person Pose Estimation from a Single RGB Image. IEEE/CVF International Conference on Computer Vision (ICCV).