ModelNet

ModelNet is a collection of 3D computer-aided design (CAD) models of everyday objects, labelled by category, built at Princeton by Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang and Jianxiong Xiao to train 3D ShapeNets, a convolutional deep belief network on voxel grids, presented at CVPR 2015 (Wu et al., 2015). The paper describes 151,128 models in 660 categories. Two subsets became benchmarks: ModelNet10, ten categories cleaned and orientation-aligned by hand, and ModelNet40, forty categories. ModelNet40 classification has been used to compare 3D deep learning methods on meshes, voxels, rendered views and point clouds, from 3D ShapeNets to PointNet and later point networks; Goyal et al. (2021) call it the most widely adopted benchmark for point-cloud classification. This article covers the construction, the release and its terms, the original classification and retrieval protocol as the 2015 paper states it, the later split and the protocol differences documented by Goyal et al. (2021), and an example that runs the original protocol on synthetic shapes.

Purpose

Wu et al. (2015, §5) argue that learning a 3D shape representation that captures the variation within a category needs a large collection of shapes, and that earlier CAD datasets such as the Princeton Shape Benchmark were limited both in the number of categories and in the number of examples per category. ModelNet was built to fill that gap. The project homepage, as of November 2026, states the goal as a comprehensive, clean collection of 3D CAD models of objects for researchers in computer vision, computer graphics, robotics and cognitive science.

Dataset Size

The paper reports 151,128 models in 660 unique categories, which it calls 22 times larger than the Princeton Shape Benchmark’s 6,670 models in 161 categories (§5). The homepage, as of November 2026, gives different totals: “127,915 CAD Models” and “662 Object Categories”, with “10 Categories with Annotated Orientation”. Neither source explains the difference; the published ShapeNet, 3D Deep Learning and PointNet articles give the paper’s figures.

The benchmark subsets are much smaller. The paper’s experiments used 40 categories with 100 unique models each (§6). The ModelNet40 split that PointNet and later point networks use has 12,311 models, 9,843 for training and 2,468 for testing, as the PointNet article reports it from Qi et al. (2017). Goyal et al. (2021, §4) describe the same benchmark as “9840 objects in the training set and 2468 in the test set”; the two training counts differ by three, and neither paper is reconciled with the other here. OctNet’s authors, as the 3D Deep Learning article reports, describe ModelNet10 as 3,991 training and 908 test shapes.

Classes

The vocabulary comes from the SUN database: Wu et al. queried common object categories of SUN that contain no fewer than 20 object instances, and removed categories with too few search results, leaving 660 (§5). ModelNet40 is the paper’s choice of 40 common object categories, and ModelNet10 a 10-category subset of them (§6, §6.1); Figure 6 of the paper shows shapes sampled from the trained 3D ShapeNets for sofa, bathtub, toilet, chair, bed, desk, table and night stand, and the supplement’s confusion matrices cover the 10 and the 40 categories.

Some ModelNet40 categories are hard to tell apart from shape alone. The supplement names cone and tent, chair and bench, plant and flower pot, and wardrobe and dresser, and argues that appearance or context, not 3D shape, separates them; this is its explanation of why the gain of 3D ShapeNets over the baselines was smaller on 40 categories than on 10.

Annotation Types

Every model carries one category label. Wu et al. (§5) describe the cleaning: models came from 3D Warehouse and from the Yobi3D search engine, which indexes 261 CAD websites, plus the Princeton Shape Benchmark. Workers on Amazon Mechanical Turk saw a sequence of thumbnails and answered yes or no to whether the category label matched each model; the authors then checked each model by hand, removed irrelevant objects from the files (a floor, a thumbnail image, a person standing next to the object) so that each mesh holds one object of its category, and discarded unrealistic models (overly simplified, or containing only images of the object) and duplicates. The homepage links the crowdsourcing interface, TurkCleaner, on the Princeton Vision Group’s code page.

ModelNet10 adds orientation: its models were cleaned in-house and their orientations “(not scale)” aligned by hand, in the homepage’s words. An orientation-aligned version of ModelNet40 was contributed later by N. Sedaghat, M. Zolfaghari, E. Amiri and T. Brox and is linked from the homepage to the lmb-freiburg/orion repository; the homepage’s own reference [8] for their BMVC paper lists three authors (Sedaghat, Zolfaghari and Brox), without Amiri. Goyal et al. (2021, §4) describe ModelNet40’s objects as aligned to a common up and front direction.

The models are polygon meshes in the Object File Format (OFF); the homepage points to MATLAB functions for reading and visualizing OFF files in the Princeton Vision Toolkit.

Splits

Both subset archives include their training and test split, according to the homepage and the 3D ShapeNets project page. Wu et al. used 100 models per category and say only that, of the 48,000 rotated models, 38,400 were for training and 9,600 for testing (§6.1); that is 80 and 20 models per category (our arithmetic). As the 3D Deep Learning article reports, Qi et al. (2016) note that 3D ShapeNets and MVCNN trained and tested on the first 80 and 20 models of each category, while VoxNet used the website’s split, so their numbers are not strictly comparable. The 9,843/2,468 split used from PointNet on is unbalanced across categories: PointNet’s overall and class-average accuracies on it differ (89.2% against 86.2%, as the PointNet article reports), which on a test set with equal counts per category they could not (our reasoning; see Evaluation).

Tasks

• Shape classification: assign one of 10 or 40 categories to a model, given as a mesh, a voxel grid, rendered views or a sampled point cloud.
• Shape retrieval: rank the other test models by similarity to a query model; a retrieved model is relevant when it has the query’s category.
• Generation and completion: Wu et al. also sampled shapes from the trained 3D ShapeNets (Figure 6) and completed shapes from single depth maps (Figure 8).

Evaluation

The 2015 protocol

Wu et al. (2015, §6, §6.1) defined the protocol that the first ModelNet results used:

• Data: 40 categories with 100 unique models each. Each model was rotated every 30° about the gravity direction, giving 12 poses and 48,000 models “with rotation enlargement”, 38,400 for training and 9,600 for testing. A 10-category subset of the 40-category data gave a smaller-scale result.
• Classification: a linear support vector machine on each feature (the fifth layer of the fine-tuned 3D ShapeNets, the Light Field descriptor with 4,700 dimensions, the Spherical Harmonic descriptor with 544), scored by average category accuracy.
• Retrieval: L2 distance between features (for the Light Field descriptor, its own distance measure; footnote 3). “Given a query from the test set, a ranked list of the remaining test data is returned”. Two measures: the mean area under the precision-recall curve (AUC) over all test queries, with each curve interpolated (footnote 4), and mean average precision (MAP), “where AP is defined as the average precision each time a positive sample is returned”. Figure 7 plots the precision-recall curves “at standard recall levels”.

Table 1 of the paper (read from the rendered page) gives:

Subset	Measure	SPH	LFD	3D ShapeNets
10 classes	classification	79.79%	79.87%	83.54%
10 classes	retrieval AUC	45.97%	51.70%	69.28%
10 classes	retrieval MAP	44.05%	49.82%	68.26%
40 classes	classification	68.23%	75.47%	77.32%
40 classes	retrieval AUC	34.47%	42.04%	49.94%
40 classes	retrieval MAP	33.26%	40.91%	49.23%

Three details matter for anyone comparing with these numbers:

• The test set is balanced. Every category has 20 models in 12 poses, 240 test items. On a balanced test set the overall accuracy $\sum_c n_c a_c / N$ equals the class-mean accuracy $\frac{1}{C}\sum_c a_c$, because every $n_c = N/C$. “Average category accuracy” and overall accuracy are therefore the same number under this protocol. PointNet’s Table 1, as the PointNet article reproduces it, lists 3D ShapeNets at 77.3% average class and 84.7% overall accuracy; on Wu et al.’s balanced test set the two would be equal for any classifier, so in our reading the two figures cannot both come from that test set. The PointNet article does not say which evaluation the 84.7 comes from, and we did not read Qi et al.’s paper to find out.
• Each test model appears 12 times. Every query in retrieval is one pose of a test model, and “the remaining test data” contains that model’s 11 other poses. The paper does not say whether they are removed from the ranked list or counted as relevant. Both baselines are rotation invariant, as the paper notes; in our reading, under such a descriptor these copies are nearly identical to the query. The example below scores both readings.
• The interpolation is unspecified. The paper does not say how each precision-recall curve is interpolated or integrated.

No evaluation code is released with the dataset. As of November 2026 neither the ModelNet homepage, the 3D ShapeNets project page nor the Princeton Vision Group’s code page links an evaluation script, so the protocol above is the paper’s text, and the example implements it from that text.

The leaderboard

The homepage keeps a ModelNet Benchmark Leaderboard, updated by email to Shuran Song with the algorithm’s name, its ModelNet40 and ModelNet10 classification and retrieval results, and the paper. The table has four columns: ModelNet40 Classification (Accuracy), ModelNet40 Retrieval (mAP), ModelNet10 Classification (Accuracy) and ModelNet10 Retrieval (mAP). The first row, 3D ShapeNets, lists 77%, 49.2%, 83.5% and 68.3%, the classification and MAP values of Table 1. As of November 2026 the highest-numbered reference on the table is RS-CNN (CVPR 2019).

The column heading says “Accuracy” without saying whether it is overall or averaged over classes, and the entries come from the authors’ own set-ups, splits and protocols: nothing in the table records them. Two entries are both called PointNet: Qi et al.’s point network (89.2% on ModelNet40) and a 3D convolutional network by Garcia-Garcia et al. (77.6% on ModelNet10).

The later protocol

PointNet, PointNet++ and Deep Sets use the 12,311-model split, as the PointNet article reports, and PointNet’s Table 1 prints both overall accuracy and class-average accuracy, which differ because that split is unbalanced: 89.2% overall and 86.2% average class for PointNet. For PointNet the authors sample 1,024 points on the mesh faces by area and normalize them into the unit sphere, with random rotation about the up axis and jitter during training (PointNet article); Goyal et al. (2021, §3.1) call 1,024 points with only $x, y, z$ coordinates the most widely adopted input scheme.

Goyal, Law, Liu, Newell and Deng (2021) showed that the factors around the network, which they call the protocol, change ModelNet40 accuracy by amounts large enough to obscure the effect of architecture (abstract). Their §3.1 separates five factors:

• Data augmentation: jitter, random rotation about the vertical axis, random scaling, random translation. Because ModelNet40 is aligned, they report that rotation augmentation hurts, and that RSCNN and DGCNN use only scaling and translation.
• Input points: a fixed set of 1,024 points per object, or a fresh sample each epoch (RSCNN, PointCNN).
• Loss: cross-entropy, or DGCNN’s smoothed labels (smooth-loss; the site’s label smoothing article covers the idea), which they find improves every architecture.
• Model selection: the final model after a number of epochs tuned on a validation set (“final model”), or the epoch with the best test accuracy (“best test model”), which DGCNN and RSCNN use.
• Ensembling: PointNet and PointNet++ average predictions over rotated and shuffled copies (“Rotation Vote”). RSCNN and DensePoint average over randomly scaled and sampled copies, repeat this 300 times on the test set and report the best accuracy (“Repeated Scaling Vote”).

Their Table 2 trains each architecture under each protocol. PointNet ranges from 89.0 ± 0.2 under the PointNet++ protocol without vote to 90.5 ± 0.1 under DGCNN’s protocol with smooth loss (90.7 for its best run); PointNet++ from 89.8 ± 0.2 to 93.3 ± 0.3; their own SimpleView, a network on six orthographic depth projections, from 90.7 ± 0.3 to 93.6 ± 0.3 (93.9 best run). In their abstract the authors conclude that such factors make differences large enough to obscure the effect of architecture, and that with the protocol controlled, PointNet++ performs competitively with recent methods. Their recommended protocol uses DGCNN’s augmentation and smooth loss but no feedback from the test set: final-model selection and no ensemble. They also report that one cited multi-view method (Ahmed et al., 2019) used a manual alignment of the test objects that differs from the standard test set, confirmed with its authors (§2). Selecting the best of many test-set evaluations is the mechanism the Benchmark Overfitting article simulates.

Example: the 2015 protocol on synthetic shapes

The script below runs the 2015 protocol on synthetic CAD-like models with exactly known categories, using no ModelNet file and training no network. It tests two things: that overall and class-mean accuracy coincide on the balanced test set, and how much retrieval scores change when the query model’s other 11 poses stay in “the remaining test data”.

• Models. Ten categories, each a triangle mesh with random proportions: box, capped cylinder, cone, ellipsoid, torus, L-block (two boxes), square pyramid, bowl (an open hemispherical shell, stretched or squashed in height), open tube and table (a top and four legs). There are 100 models per category; models 0–79 are for training and 80–99 for testing, the first-80/20 split.
• Poses. Each model is rotated by $30k°$ about $z$, $k = 0, \dots, 11$. For each pose, 512 points are sampled uniformly on the surface (a triangle chosen with probability proportional to its area, then a uniform point inside it), centred on their mean and scaled so the farthest point lies at distance 1. A fresh sample per pose is our analogue of the small differences that discretization introduces between rotated copies of one mesh; the retrieval outcome tests this analogue, not ModelNet’s rendered or voxelized copies. The test set has 2,400 items, 240 per category, the same per-category counts as Wu et al.’s test set.
• Descriptor. Our descriptor is the histogram of all 130,816 pairwise distances between the 512 points, in 32 bins on $[0, 2]$, normalized to sum to 1. It depends only on distances between points, so it is exactly invariant to rotation, like the two descriptors of 2015.
• Classification. A 1-nearest-neighbour classifier with L2 distance between histograms stands in for the paper’s linear SVM. Overall accuracy (OA) and class-mean accuracy (mAcc) are computed as exact fractions. One further line scores the same predictions on an unbalanced subset that we chose: 20, 18, …, 2 test models in the ten categories, pose 0 only.
• Retrieval. Every test item is a query, the others are ranked by L2 distance (ties broken by item index), and an item is relevant if it has the query’s category. Copies in keeps the query model’s 11 other poses in the list (239 relevant items, 11 of them copies); copies out removes them (228 relevant). Unlike the leakage between training and test data that the Training, Validation, and Test Sets article simulates, no copy crosses into the training set here: the copies sit inside the test gallery that each query is ranked against. MAP is the non-interpolated average precision, the mean precision at each relevant item. AUC is printed in two readings: AUC_step, the exact area under the interpolated curve $p_{\text{interp}}(r) = \max_{r' \ge r} p(r')$, and AUC_11, the trapezoidal area of $p_{\text{interp}}$ at recall $0, 0.1, \dots, 1$.
• Seeds. The code loops over data seeds 0, 1 and 2; each regenerates all models and samples.

The plan, written before the pilot run, fixed these checks (C) and one hypothesis (H):

• C1: on the balanced test set, OA equals mAcc exactly, as fractions.
• C2: an oracle ranking (relevant items first) gives MAP, AUC_step and AUC_11 equal to 1 in both conditions.
• C3: our AP equals scikit-learn’s average_precision_score within $10^{-12}$ on every query with no tied distances.
• C4: the descriptor of a sample equals that of its 12 rotations.
• C5: the generator gives the planned counts: 9,600 training and 2,400 test items, 239 and 228 relevant items per query, 11 same-model items in every copies-in list, farthest point at distance 1; on a hand-made list of three items with its 2 relevant items at ranks 1 and 3, AP is 5/6 and AUC_11 is 0.85.
• C6: when the 11 copies occupy ranks 1–11, $\mathrm{AP}_{\text{in}} \ge (11 + 228\,\mathrm{AP}_{\text{out}})/239$. The derivation: removing the copies leaves the order of the other items unchanged, so the $j$-th of the 228 other relevant items, at rank $\rho_j \ge j$ in the copies-out list, sits at rank $\rho_j + 11$ with $j + 11$ hits in the copies-in list, and $(j+11)/(\rho_j+11) \ge j/\rho_j$; the 11 copies each add precision 1.
• L: the task is learnable but not trivial: balanced OA above chance (0.1) and below 1.
• H1: in every seed, copies in gives a higher MAP, AUC_step and AUC_11 than copies out.

H1 was written after a planning script, which ran the same set-up before the plan was fixed, had printed the gains, so the example is an illustration rather than a test of a prediction. The comparison with scikit-learn is limited to queries without tied distances because a planning diagnostic found exactly tied distances on some queries, where scikit-learn’s tie handling (one threshold for tied scores) and our index tie-break legitimately differ. After the pilot, one change was approved before the first run: the wording of the C3 line’s count, which now says that it counts query–condition pairs. Nothing else changed between the pilot and the first run.

The script uses NumPy 2.3.5, SciPy 1.16.3 and scikit-learn 1.7.2 with one BLAS thread, and took about 35–37 seconds on a laptop CPU in three runs; the output was identical on a second run.

import os
os.environ["OMP_NUM_THREADS"] = "1"; os.environ["OPENBLAS_NUM_THREADS"] = "1"
os.environ["MKL_NUM_THREADS"] = "1"
from fractions import Fraction
import numpy as np
from scipy.spatial.distance import pdist, cdist
from sklearn.metrics import average_precision_score

N_CLASS, N_MODEL, N_TRAIN, N_POSE, N_PTS, N_BINS = 10, 100, 80, 12, 512, 32
N_TEST = N_MODEL - N_TRAIN
CLASSES = ["box", "cylinder", "cone", "ellipsoid", "torus", "l_block", "pyramid", "bowl",
           "tube", "table"]

# ------------------------------------------------------------ synthetic CAD models (triangles)
def grid_tris(P):
    """P: (nu+1, nv+1, 3) parametric grid -> (2*nu*nv, 3, 3) triangles."""
    a, b, c, d = P[:-1, :-1], P[1:, :-1], P[1:, 1:], P[:-1, 1:]
    return np.concatenate([np.stack([a, b, c], -2).reshape(-1, 3, 3),
                           np.stack([a, c, d], -2).reshape(-1, 3, 3)])

def box_tris(lo, hi):
    x0, y0, z0 = lo; x1, y1, z1 = hi
    v = np.array([[x0, y0, z0], [x1, y0, z0], [x1, y1, z0], [x0, y1, z0],
                  [x0, y0, z1], [x1, y0, z1], [x1, y1, z1], [x0, y1, z1]])
    f = [(0, 1, 2, 3), (4, 5, 6, 7), (0, 1, 5, 4), (1, 2, 6, 5), (2, 3, 7, 6), (3, 0, 4, 7)]
    return np.array([[v[q[0]], v[q[1]], v[q[2]]] for q in f] + [[v[q[0]], v[q[2]], v[q[3]]] for q in f])

def disk_tris(r, z, nth=48, nr=4):
    t = np.linspace(0, 2 * np.pi, nth + 1); s = np.linspace(0, r, nr + 1)
    S, T = np.meshgrid(s, t, indexing="ij")
    return grid_tris(np.stack([S * np.cos(T), S * np.sin(T), np.full_like(S, z)], -1))

def revolve(rad, z, nth=48):
    """Surface of revolution through profile points (rad[i], z[i])."""
    t = np.linspace(0, 2 * np.pi, nth + 1)
    R, T = np.meshgrid(rad, t, indexing="ij"); Z = np.meshgrid(z, t, indexing="ij")[0]
    return grid_tris(np.stack([R * np.cos(T), R * np.sin(T), Z], -1))

def make_mesh(cls, rng):
    u = lambda lo, hi: rng.uniform(lo, hi)
    if cls == "box":
        a, b, c = u(0.5, 1.5), u(0.5, 1.5), u(0.5, 1.5)
        return box_tris((0, 0, 0), (a, b, c))
    if cls == "cylinder":
        r, h = u(0.3, 0.8), u(0.6, 2.0)
        return np.concatenate([revolve(np.full(9, r), np.linspace(0, h, 9)),
                               disk_tris(r, 0.0), disk_tris(r, h)])
    if cls == "cone":
        r, h = u(0.3, 0.8), u(0.6, 2.0)
        t = np.linspace(0, 1, 9)
        return np.concatenate([revolve(r * (1 - t), h * t), disk_tris(r, 0.0)])
    if cls == "ellipsoid":
        a, b, c = u(0.4, 1.2), u(0.4, 1.2), u(0.4, 1.2)
        th = np.linspace(0, np.pi, 25); ph = np.linspace(0, 2 * np.pi, 49)
        TH, PH = np.meshgrid(th, ph, indexing="ij")
        return grid_tris(np.stack([a * np.sin(TH) * np.cos(PH), b * np.sin(TH) * np.sin(PH),
                                   c * np.cos(TH)], -1))
    if cls == "torus":
        R, r = u(0.6, 1.0), u(0.15, 0.45)
        a = np.linspace(0, 2 * np.pi, 49); b = np.linspace(0, 2 * np.pi, 25)
        A, B = np.meshgrid(a, b, indexing="ij")
        return grid_tris(np.stack([(R + r * np.cos(B)) * np.cos(A), (R + r * np.cos(B)) * np.sin(A),
                                   r * np.sin(B)], -1))
    if cls == "l_block":
        a, b, c = u(1.0, 1.8), u(0.3, 0.6), u(0.3, 0.6)
        h = u(0.8, 1.6)
        return np.concatenate([box_tris((0, 0, 0), (a, b, c)), box_tris((0, 0, c), (b, b, c + h))])
    if cls == "pyramid":
        a, h = u(0.6, 1.4), u(0.5, 1.5)
        q = np.array([[0, 0, 0], [a, 0, 0], [a, a, 0], [0, a, 0]], float); apex = np.array([a / 2, a / 2, h])
        side = [[q[i], q[(i + 1) % 4], apex] for i in range(4)]
        return np.array(side + [[q[0], q[1], q[2]], [q[0], q[2], q[3]]])
    if cls == "bowl":
        r, s = u(0.5, 1.0), u(0.4, 1.2)
        th = np.linspace(np.pi / 2, np.pi, 13)
        return revolve(r * np.sin(th), s * r * np.cos(th))
    if cls == "tube":
        r, h = u(0.3, 0.8), u(0.6, 2.0)
        return revolve(np.full(9, r), np.linspace(0, h, 9))
    if cls == "table":
        w, d, t, H, l = u(0.8, 1.6), u(0.6, 1.2), u(0.04, 0.12), u(0.5, 1.0), u(0.05, 0.12)
        parts = [box_tris((0, 0, H), (w, d, H + t))]
        for x0, y0 in [(0, 0), (w - l, 0), (0, d - l), (w - l, d - l)]:
            parts.append(box_tris((x0, y0, 0), (x0 + l, y0 + l, H)))
        return np.concatenate(parts)
    raise ValueError(cls)

def sample_surface(tris, n, rng):
    """n points uniformly on the surface: triangles by area, then uniform barycentric."""
    area = 0.5 * np.linalg.norm(np.cross(tris[:, 1] - tris[:, 0], tris[:, 2] - tris[:, 0]), axis=1)
    idx = rng.choice(len(tris), size=n, p=area / area.sum())
    r1, r2 = np.sqrt(rng.random(n)), rng.random(n)
    t = tris[idx]
    return (1 - r1)[:, None] * t[:, 0] + (r1 * (1 - r2))[:, None] * t[:, 1] + (r1 * r2)[:, None] * t[:, 2]

def rot_z(deg):
    c, s = np.cos(np.radians(deg)), np.sin(np.radians(deg))
    return np.array([[c, -s, 0], [s, c, 0], [0, 0, 1]])

def normalize(p):
    p = p - p.mean(0)
    return p / np.linalg.norm(p, axis=1).max()

def d2(p):
    """Our descriptor: histogram of all pairwise point distances, 32 bins on [0, 2]."""
    h, _ = np.histogram(pdist(p), bins=N_BINS, range=(0.0, 2.0))
    return h / h.sum()

def build(seed):
    """Descriptors X[class, model, pose]; each of the 12 poses is a fresh surface sample."""
    rng = np.random.default_rng(seed)
    X = np.zeros((N_CLASS, N_MODEL, N_POSE, N_BINS)); norm_err = 0.0
    for c, name in enumerate(CLASSES):
        for m in range(N_MODEL):
            tris = make_mesh(name, rng)
            for k in range(N_POSE):
                p = normalize(sample_surface(tris, N_PTS, rng) @ rot_z(30 * k).T)
                norm_err = max(norm_err, abs(np.linalg.norm(p, axis=1).max() - 1))
                X[c, m, k] = d2(p)
    return X, norm_err

# ------------------------------------------------------------ retrieval measures
def ap_curve(rel):
    """rel: relevance in ranked order -> (AP, AUC_step, AUC_11)."""
    hits = np.flatnonzero(rel) + 1; R = len(hits)
    prec = np.arange(1, R + 1) / hits                    # precision at each relevant item
    ap = prec.mean()                                     # non-interpolated AP
    interp = np.maximum.accumulate(prec[::-1])[::-1]     # max precision at recall >= r
    auc_step = interp.mean()                             # exact area under interpolated curve
    rec = np.arange(1, R + 1) / R
    lv = np.linspace(0, 1, 11)
    pi = np.array([interp[np.searchsorted(rec, r - 1e-12)] for r in lv])
    return ap, auc_step, np.trapezoid(pi, lv)            # trapezoid over 11 recall levels

def gallery(q, model, copies_in):
    keep = np.arange(len(model)) != q
    if not copies_in:
        keep &= model != model[q]                        # drop the query model's other poses
    return np.flatnonzero(keep)

def retrieval(D, y, model, copies_in):
    """D: distances (smaller = more similar). Ties are broken by item index."""
    res, orders = [], []
    for q in range(len(y)):
        idx = gallery(q, model, copies_in)
        order = idx[np.argsort(D[q, idx], kind="stable")]
        res.append(ap_curve(y[order] == y[q])); orders.append(order)
    return np.array(res), orders

# ------------------------------------------------------------ one seed
def run(seed):
    X, norm_err = build(seed)
    Xtr = X[:, :N_TRAIN].reshape(-1, N_BINS); Xte = X[:, N_TRAIN:].reshape(-1, N_BINS)
    ytr = np.repeat(np.arange(N_CLASS), N_TRAIN * N_POSE)
    yte = np.repeat(np.arange(N_CLASS), N_TEST * N_POSE)
    mte = np.repeat(np.arange(N_CLASS * N_TEST), N_POSE)        # test model id of each item
    print(f"seed {seed}: train {len(ytr)}, test {len(yte)} items; max |farthest norm - 1| {norm_err:.1e}")

    # (1) original balanced protocol: 1-NN, overall against class-mean accuracy
    correct = ytr[np.argmin(cdist(Xte, Xtr), axis=1)] == yte
    oa = Fraction(int(correct.sum()), len(yte))
    macc = sum(Fraction(int(correct[yte == c].sum()), int((yte == c).sum()))
               for c in range(N_CLASS)) / N_CLASS
    print(f"  balanced (240 per class): OA {float(oa):.4f}  mAcc {float(macc):.4f}  "
          f"C1 exactly equal: {oa == macc}")
    counts = [20, 18, 16, 14, 12, 10, 8, 6, 4, 2]                # synthetic, pose 0 only
    sel = np.zeros(len(yte), bool)
    for c, n in enumerate(counts):
        sel[(c * N_TEST + np.arange(n)) * N_POSE] = True
    ca, ya = correct[sel], yte[sel]
    print(f"  unbalanced {counts}: OA {ca.mean():.4f}  mAcc "
          f"{np.mean([ca[ya == c].mean() for c in range(N_CLASS)]):.4f}")

    # (2) retrieval over the remaining test data, the query model's 11 other poses in or out
    D = cdist(Xte, Xte)
    res = {ci: retrieval(D, yte, mte, ci) for ci in (True, False)}
    R_in, R_out = ({int((yte[gallery(q, mte, ci)] == yte[q]).sum()) for q in range(len(yte))}
                   for ci in (True, False))
    assert len(R_in) == len(R_out) == 1
    R_in, R_out = R_in.pop(), R_out.pop()
    n_copies = {int((mte[gallery(q, mte, True)] == mte[q]).sum()) for q in range(len(yte))}
    print(f"  relevant per query: copies in {R_in}, copies out {R_out}; same-model items "
          f"in the copies-in gallery: {sorted(n_copies)}")
    print("               MAP     AUC_step  AUC_11")
    for ci, name in [(True, "copies in "), (False, "copies out")]:
        m = res[ci][0].mean(0)
        print(f"  {name}  {m[0]:.4f}   {m[1]:.4f}    {m[2]:.4f}")
    g = res[True][0].mean(0) - res[False][0].mean(0)
    print(f"  gain        {g[0]:.4f}   {g[1]:.4f}    {g[2]:.4f}   H1 all > 0: {bool(np.all(g > 0))}")

    # checks
    ap_in, ap_out = res[True][0][:, 0], res[False][0][:, 0]
    top = np.array([np.all(mte[o[:11]] == mte[q]) for q, o in enumerate(res[True][1])])
    bound = (11 + R_out * ap_out) / (R_out + 11)
    print(f"  C6 queries with the 11 copies at ranks 1-11: {top.sum()}; "
          f"bound violations: {int(np.sum(ap_in[top] < bound[top] - 1e-12))}")
    n_tied, md = 0, 0.0
    for ci in (True, False):
        for q in range(len(yte)):
            idx = gallery(q, mte, ci); d = D[q, idx]
            if len(np.unique(d)) < len(d):
                n_tied += 1; continue
            md = max(md, abs(average_precision_score(yte[idx] == yte[q], -d) - res[ci][0][q, 0]))
    print(f"  C3 max |AP - sklearn AP| on untied queries {md:.1e} (tied query–condition pairs skipped: {n_tied})")
    S = (yte[:, None] != yte[None, :]).astype(float)              # oracle: class indicator
    oe = max(np.abs(retrieval(S, yte, mte, ci)[0] - 1).max() for ci in (True, False))
    print(f"  C2 oracle max |MAP, AUC_step, AUC_11 - 1| {oe:.1e}")

def checks():
    a = ap_curve(np.array([1, 0, 1], bool))                       # hits at ranks 1 and 3 of 2
    print(f"C5 hand list: |AP - 5/6| {abs(a[0] - 5 / 6):.1e}, |AUC_11 - 0.85| {abs(a[2] - 0.85):.1e}")
    rng = np.random.default_rng(99); p = sample_surface(make_mesh("table", rng), N_PTS, rng)
    dd = max(np.abs(d2(normalize(p @ rot_z(30 * k).T)) - d2(normalize(p))).max() for k in range(12))
    print(f"C4 D2 of a sample against its 12 rotations: max diff {dd:.1e}")

checks()
for seed in (0, 1, 2):
    run(seed)

C5 hand list: |AP - 5/6| 1.1e-16, |AUC_11 - 0.85| 0.0e+00
C4 D2 of a sample against its 12 rotations: max diff 0.0e+00
seed 0: train 9600, test 2400 items; max |farthest norm - 1| 2.2e-16
  balanced (240 per class): OA 0.9892  mAcc 0.9892  C1 exactly equal: True
  unbalanced [20, 18, 16, 14, 12, 10, 8, 6, 4, 2]: OA 0.9818  mAcc 0.9804
  relevant per query: copies in 239, copies out 228; same-model items in the copies-in gallery: [11]
               MAP     AUC_step  AUC_11
  copies in   0.5302   0.5374    0.5353
  copies out  0.4971   0.5063    0.5068
  gain        0.0331   0.0311    0.0285   H1 all > 0: True
  C6 queries with the 11 copies at ranks 1-11: 104; bound violations: 0
  C3 max |AP - sklearn AP| on untied queries 3.3e-16 (tied query–condition pairs skipped: 28)
  C2 oracle max |MAP, AUC_step, AUC_11 - 1| 0.0e+00
seed 1: train 9600, test 2400 items; max |farthest norm - 1| 3.3e-16
  balanced (240 per class): OA 0.9933  mAcc 0.9933  C1 exactly equal: True
  unbalanced [20, 18, 16, 14, 12, 10, 8, 6, 4, 2]: OA 0.9909  mAcc 0.9938
  relevant per query: copies in 239, copies out 228; same-model items in the copies-in gallery: [11]
               MAP     AUC_step  AUC_11
  copies in   0.5625   0.5701    0.5680
  copies out  0.5310   0.5408    0.5421
  gain        0.0315   0.0293    0.0259   H1 all > 0: True
  C6 queries with the 11 copies at ranks 1-11: 126; bound violations: 0
  C3 max |AP - sklearn AP| on untied queries 3.3e-16 (tied query–condition pairs skipped: 40)
  C2 oracle max |MAP, AUC_step, AUC_11 - 1| 0.0e+00
seed 2: train 9600, test 2400 items; max |farthest norm - 1| 3.3e-16
  balanced (240 per class): OA 0.9921  mAcc 0.9921  C1 exactly equal: True
  unbalanced [20, 18, 16, 14, 12, 10, 8, 6, 4, 2]: OA 0.9909  mAcc 0.9938
  relevant per query: copies in 239, copies out 228; same-model items in the copies-in gallery: [11]
               MAP     AUC_step  AUC_11
  copies in   0.5245   0.5323    0.5313
  copies out  0.4900   0.5003    0.5028
  gain        0.0344   0.0320    0.0285   H1 all > 0: True
  C6 queries with the 11 copies at ranks 1-11: 135; bound violations: 0
  C3 max |AP - sklearn AP| on untied queries 3.3e-16 (tied query–condition pairs skipped: 38)
  C2 oracle max |MAP, AUC_step, AUC_11 - 1| 0.0e+00

Accuracy. C1 held in all three seeds: on the balanced test set OA and mAcc are the same fraction (0.9892, 0.9933, 0.9921). This is the identity above, not a property of the classifier. L held: accuracy is above chance and below 1, but at about 0.99 the classification task is close to trivial for this descriptor and these shapes; we report it as printed and did not change the shapes. On the unbalanced subset the two averages separate, and the sign of the difference is not fixed: OA is above mAcc in seed 0 (0.9818 against 0.9804) and below it in seeds 1 and 2 (0.9909 against 0.9938). The Accuracy article explains how class weights produce such differences; with this few errors, the printed values show a difference of either sign on a skewed test set.

Retrieval. H1 held in all three seeds and for all three measures: keeping the copies raises MAP by 0.0331, 0.0315 and 0.0344, AUC_step by 0.0311, 0.0293 and 0.0320, and AUC_11 by 0.0285, 0.0259 and 0.0285. The leak is modest: about three points. In only 104, 126 and 135 of the 2,400 queries do the 11 copies take all of ranks 1–11, so near-identical copies are not always the nearest items under this descriptor. In our reading, one reason the gain is modest comes from construction: only 11 of the 239 relevant items are copies, the same proportion as in Wu et al.’s test set. When all copies come first, C6 gives a gain of at least $11(1 - \mathrm{AP}_{\text{out}})/239$, which is about 0.023 at the printed copies-out MAP of seed 0 (our arithmetic), the same order as the printed gains. In this analogue, then, the retrieval score depends by about three points on a choice the 2015 paper leaves open, and the two AUC readings differ from each other by at most 0.0025 in the printed output.

Checks. C2 held (the oracle reaches 1 exactly in both conditions), C3 held (largest difference from scikit-learn $3.3 \times 10^{-16}$; 28, 40 and 38 tied query–condition pairs skipped), C4 held (difference 0), C5 held (all counts as planned; the hand list matches 5/6 and 0.85 to rounding), and C6 held (no violation in any seed).

License

ModelNet has no named licence. The homepage’s copyright statement, as of November 2026, says that all CAD models were downloaded from the Internet and that their original authors hold the copyright; that the labels were obtained by the authors through Amazon Mechanical Turk and are provided freely; and that “This dataset is provided for the convenience of academic research only.” The ModelNet10 and ModelNet40 archives are linked directly from the homepage and the project page; the full dataset is obtained by emailing Shuran Song for a MATLAB toolbox that downloads it. The homepage asks users to cite Wu et al. (2015).

Limitations and Bias

• Synthetic, clean, aligned shapes. The models are complete CAD meshes, one object per file, mostly aligned. Goyal et al. (2021) contrast ModelNet40 with ScanObjectNN, whose objects come from real 3D scans with occlusion, background points and holes and are not axis-aligned.
• Categories separable only by context. The 3D ShapeNets supplement attributes part of the difficulty of 40 categories to pairs such as plant and flower pot that differ in appearance or context more than in shape.
• Copyright of the models. The models remain the property of their original authors (License).
• Protocol drift. The 2015 protocol (balanced, 12 poses, average category accuracy, retrieval over remaining test data) and the later one (the 9,843/2,468 split, overall and class-average accuracy) give numbers that are not comparable, and the leaderboard does not record which was used. Goyal et al. show that augmentation, loss, model selection and voting change accuracy by amounts large enough to obscure the effect of architecture.
• Test-set feedback. Best-test-epoch selection and the best of 300 voting runs use the test set to choose the reported number (Goyal et al., 2021, §3.1).
• Test-set size. In our reading, with 2,468 test models one model is about 0.04 percentage points of overall accuracy, so differences of a few tenths of a point rest on a handful of models.

Common Uses

ModelNet40 is used to compare 3D classification networks across representations, as the 3D Deep Learning and PointNet articles describe, and ModelNet10 as a smaller, aligned variant. Its CAD models are also sampled into partial scans for learned point cloud registration, including PREDATOR’s lower-overlap variant ModelLoNet, as that article reports; rotated by a learning agent for active perception; and, in the 2015 paper itself, used for shape generation and completion.

Historical Milestones

• 2015, ModelNet and 3D ShapeNets. Wu et al. released ModelNet and reported 77.32% average category accuracy on 40 categories and 83.54% on 10, with retrieval MAP of 49.23% and 68.26%, against the Light Field and Spherical Harmonic descriptors (Table 1).
• 2015–2016, views and voxels. MVCNN, VoxNet and the volumetric and multi-view networks of Qi et al. (2016) compared rendered views with voxel grids on ModelNet40, as the 3D Deep Learning article describes.
• 2017, PointNet. Qi et al. reported 89.2% overall and 86.2% average class accuracy on ModelNet40 from 1,024 points, on the 9,843/2,468 split (Table 1, as the PointNet article reports); Goyal et al. (2021) later call ModelNet40 the most widely adopted benchmark for point-cloud classification.
• 2021, protocol controlled. Goyal et al. retrained four point networks and SimpleView under six protocol variants (Table 2) and showed that protocol differences are large enough to obscure the effect of architecture.

Related Datasets

• ShapeNet (2015) is the larger CAD repository organized by WordNet synsets, gathered partly from the same sources; its part labels serve segmentation.
• The Princeton Shape Benchmark (6,670 models in 161 categories, as Wu et al. report; the ShapeNet article gives Chang et al.’s figure of about 1,800 models in 90 categories) was the earlier retrieval benchmark; part of ModelNet comes from it.
• Aligned ModelNet40 (Sedaghat, Zolfaghari, Amiri and Brox) adds a canonical orientation to the 40-category subset.
• ScanObjectNN is a real-scan classification set of about 15,000 objects in 15 categories, 11 of them shared with ModelNet40 (Goyal et al., 2021, §4).
• ModelLoNet is a lower-overlap registration benchmark built from ModelNet40 CAD models (Point Cloud Registration).

Resources

• Homepage, downloads, leaderboard and copyright statement: modelnet.cs.princeton.edu (as of November 2026).
• 3D ShapeNets project page, with the paper, supplement, slides and poster: 3dvision.princeton.edu/projects/2014/3DShapeNets.
• Aligned ModelNet40: github.com/lmb-freiburg/orion, as linked from the homepage.
• Goyal et al.’s code for the protocols and SimpleView: github.com/princeton-vl/SimpleView, as given in their paper.
• Current results are on the homepage’s leaderboard, with the caveats above about mixed protocols.